Download:
pdf |
pdfU.S. Department
of Labor
Employment
and Training
Administration
O*NET® Data
Collection Program
Office of Management and Budget
Clearance Package Supporting Statement
Part A: Justification
with
Table of Contents
List of exhibits
References
April 10, 2012
Table of Contents
Section
A.
Page
Justification ....................................................................................................................................... 1
A.1
A.2
A.3
A.4
A.5
A.6
A.7
A.8
A.9
A.10
A.11
A.12
A.13
A.14
A.15
A.16
A.17
A.18
Circumstances of Information Collection ................................................................................... 1
A.1.1 What Is the O*NET Program? ...................................................................................... 2
A.1.2 The O*NET Data Collection Approach ......................................................................... 5
A.1.3 Summary of the O*NET Data Collection Process ........................................................ 9
A.1.4 Summary of Response Rate Experience to Date....................................................... 12
A.1.5 Statutory and Regulatory Information ......................................................................... 16
A.1.6 Federal Register Notice .............................................................................................. 23
Uses, Products, and Services Based on the O*NET Program ............................................... 23
A.2.1 The O*NET Database, O*NET OnLine, My Next Move, O*NET Career Tools,
O*NET Training Academy, and O*NET Code Connector .......................................... 24
A.2.2 O*NET Web Site Statistics ......................................................................................... 27
A.2.3 Examples of O*NET Data and Products in Use ......................................................... 30
A.2.4 Examples of the O*NET Program in Published Literature ......................................... 47
A.2.5 Looking to the Future with Competency Models ........................................................ 48
Uses of Information Technology ............................................................................................. 49
A.3.1 Web Questionnaires ................................................................................................... 50
A.3.2 Project Web Site ......................................................................................................... 50
A.3.3 The Case Management System and Data Collection Utilities .................................... 52
A.3.4 Additional Uses of the Internet for Data Collection .................................................... 53
Efforts to Identify Duplication................................................................................................... 54
Efforts to Minimize Burden on Small Establishments ............................................................. 55
Consequences of Collecting the Information Less Frequently ................................................ 56
Special Circumstances ............................................................................................................ 57
Consultation Outside the Agency ............................................................................................ 58
Payments or Gifts to Respondents ......................................................................................... 59
A.9.1 Incentives for the Point of Contact and the Employer ................................................ 59
A.9.2 Incentives for the Employee ....................................................................................... 60
A.9.3 Incentives for Occupation Experts .............................................................................. 61
Assurance of Privacy............................................................................................................... 61
Questions of a Sensitive Nature .............................................................................................. 63
Estimates of Annualized Hour Burden .................................................................................... 63
Annual Reporting Burden Cost ............................................................................................... 68
Estimates of Annualized Cost to Government ........................................................................ 68
Reasons for Program Changes or Adjustments Reported in Sections A.13 and A.14 ........... 68
Time Schedule, Publication, and Analysis Plans .................................................................... 69
A.16.1 Data Analysis Tasks Conducted for Each Cycle ........................................................ 69
A.16.2 Creation of the Occupation Database ........................................................................ 73
Display of Expiration Date ....................................................................................................... 73
Exceptions to Certification Statement ..................................................................................... 73
iii
B.
Collection of Information Employing Statistical Methods .......................................................... 75
B.1
B.2
B.3
B.4
B.5
C.
iv
Sampling Universe, Sampling Methods, and Expected Response Rates .............................. 75
B.1.1 Establishment Method ................................................................................................ 75
B.1.2 Occupation Expert Method ....................................................................................... 100
Procedures for the Collection of Information ......................................................................... 101
B.2.1 Establishment Method .............................................................................................. 102
B.2.2 Occupation Expert Method ....................................................................................... 108
Methods to Maximize Response Rates ................................................................................. 110
Tests of Procedures .............................................................................................................. 112
B.4.1 1999 Pretest Experiments ........................................................................................ 112
B.4.2 Wave 1.1 Experiment ............................................................................................... 112
B.4.3 Point-of-Contact Incentive Experiment ..................................................................... 113
B.4.4 Experiments in Weight Trimming ............................................................................. 113
B.4.5 Experiments in Model-Aided Sampling .................................................................... 114
B.4.6 Alternative Measures of Uncertainty ........................................................................ 114
B.4.7 Suppression of Estimates with Poor Precision ......................................................... 115
B.4.8 Dual-Frame Sampling for Hard-to-Find Occupations ............................................... 115
B.4.9 Alternative Levels of Population Coverage .............................................................. 116
B.4.10 Adaptive Total Design .............................................................................................. 116
B.4.11 Analysis of Unit and Item Nonresponse ................................................................... 116
B.4.12 Additional Tests of Procedures ................................................................................ 117
Statistical Consultants ........................................................................................................... 117
References .................................................................................................................................... 119
List of Exhibits
Number
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
Page
O*NET Content Model ...................................................................................................................... 3
O*NET Data Collection Program Questionnaires ............................................................................ 4
Establishment Method Data Collection Results ............................................................................. 12
Occupation Expert Method Data Collection Results ...................................................................... 15
O*NET Citations in Code of Federal Regulations .......................................................................... 18
Database Updates .......................................................................................................................... 24
Main Organization Types Submitting O*NET Certifications ........................................................... 29
O*NET Product Downloads ............................................................................................................ 29
Distribution of Frame and Sample Establishments by Employment Size ...................................... 56
Estimate of Hour and Cost Burden by Year ................................................................................... 66
Comparison of Hour and Cost Burden Between 2009–2011 and June 2012–May 2015 .............. 70
Data Analysis and Publication Schedule ........................................................................................ 71
Summary of Sample Selection Process ......................................................................................... 80
Half-Width of 95% Confidence Intervals ........................................................................................ 88
Classification of Establishments by Occupation Model-Aided Sampling Status ............................ 93
Overlap of Full and Supplemental Frames..................................................................................... 96
Establishment Method Data Collection Flowchart........................................................................ 104
Occupation Expert Method Data Collection Flowchart ................................................................ 110
Statistical Consultants .................................................................................................................. 118
v
A. Justification
A.1
Circumstances of Information Collection
This Supporting Statement is a request and justification for a 3-year clearance from the
Office of Management and Budget (OMB) to continue the Occupational Information Network
(O*NET®) Data Collection Program. The O*NET Data Collection Program continually operates
to populate and maintain a current database on the detailed characteristics of workers,
occupations, and skills. The program uses an occupational taxonomy, the O*NET-SOC, which is
based on the 2010 version of the Standard Occupational Classification (SOC) mandated by OMB
for use by all federal agencies collecting occupational and labor market information. The
O*NET-SOC comprises occupations at the most detailed level of the SOC and includes
additional occupational specificity as needed. In addition, new and emerging occupations in
high-growth sectors of the economy are being identified and added to the taxonomy. Data have
been published for 874 O*NET-SOC occupations, 359 of which have undergone a second, more
recent update. The O*NET Data Collection Program received initial OMB clearance in 1999 for
a pretest and four subsequent clearances that have allowed main study data collection to continue
without interruption since June 2001. Our current clearance expires May 31, 2012. This request
is to continue to collect data on selected high-growth and new and emerging occupations for
three more years (June 2012–May 2015), subject to annual budget levels.
The continued population of the O*NET database is important because the O*NET
database is the most current and comprehensive standard source of occupational information in
the United States. The O*NET Data Collection Program remains at the center of an extensive
network of occupational and skill information used by a wide range of audiences, including
individuals making career decisions; public agencies, such as workforce investment boards
(WIBs) and One-Stop Career Centers, making training investment decisions; educational
institutions preparing a future workforce; and employers making staffing and training decisions.
This program provides a common language and framework to facilitate communication
about industry skill needs among business, education, and the workforce investment system. The
resulting O*NET database also is used to develop industry competency models and occupational
competency profiles. O*NET data include information about transferable skills that are used for
skills gap analysis, facilitating a mobile workforce responsive to changing economic needs. The
O*NET database and companion O*NET Career Exploration Tools are used by many private
companies and public organizations to tailor applications to their needs and those of their
1
Justification
OMB Supporting Statement
customers. The broad utility of the O*NET tools contributes to U.S. competitiveness in a global,
21st-century economy.1
The O*NET Data Collection Program employs a multiple-method approach to updating
the O*NET database. The primary method involves a two-stage sample design to survey
establishments and workers in those establishments. When necessary, this method may be
supplemented with a sample selected from additional sources, such as professional and trade
association membership lists, resulting in a dual-frame approach. An alternative method, based
on sampling from lists of identified occupation experts, is used for occupations for which the
primary method is inefficient. This method is reserved for selected occupations, such as those
with small employment scattered among many industries and those for which no employment
data currently exist on which to base a sample, such as new and emerging occupations. With all
methods, the O*NET survey instruments are used.2 The rest of Section A describes the O*NET
Program and reviews statutory and regulatory information.
A.1.1 What Is the O*NET Program?
The O*NET Program is a comprehensive system for collecting and disseminating
information on occupational and worker requirements. The O*NET data supersede the U.S.
Department of Labor’s (DOL’s) Dictionary of Occupational Titles (DOT) and provide additional
occupational requirements not available in the DOT. The DOT is no longer updated or
maintained by DOL.
As shown in Exhibit 1, the O*NET Program uses a data structure, the Content Model, to
organize occupational information and to provide a common language of standardized and
defined occupation descriptors and measures for use by all audiences. The O*NET Content
Model is the result of extensive research, and its development is fully documented (Peterson,
Mumford, Borman, Jeanneret, & Fleishman, 1995, pp. 2–6; Peterson, Mumford, Borman, et al.,
1997; Peterson et al., 2001). It comprises worker-oriented and job-oriented characteristics at both
an occupation-specific level and across occupations, as the exhibit illustrates.
Each of the six domains of the Content Model groups information hierarchically. For
example, the Worker Characteristics domain contains four types of information: Abilities,
Occupational Interests, Work Values, and Work Styles. From these four, the Abilities domain, in
turn, contains four types of abilities: Cognitive, Psychomotor, Physical, and Sensory. Each of
1
For details on the uses of the O*NET Program, see Section A.2. More information about the O*NET Data
Collection Program can be found at the National O*NET Program’s public Web site, http://www.onetcenter.org/
(the O*NET portal page that links to several O*NET-related Web sites) and at http://www.doleta.gov/programs/
onet (Web site of the Employment and Training Administration at the U.S. Department of Labor).
2
For detailed information on these methods, see Sections B.1 and B.2.
2
OMB Supporting Statement
Justification
Exhibit 1. O*NET Content Model
these types of abilities contains further levels of detail. For example, the Psychomotor type
includes Fine Manipulative, Control Movement, and Reaction Time and Speed. Finally, Fine
Manipulative contains three specific descriptors: Arm-Hand Steadiness, Manual Dexterity, and
Finger Dexterity. Hierarchies both organize the information and allow for different levels of
specificity. By organizing occupational information hierarchically, the O*NET taxonomies of
information and SOC-based occupations facilitate the use of a common language to describe the
world of work.
The O*NET Program takes the best knowledge about both content and methodology
from the last 70 years of research, since the first DOT. The descriptors and rating scales for
O*NET data were developed through extensive research, drawing primarily from job analysis in
industrial/organizational psychology and human resource management (Peterson et al., 1995).
The descriptors in the O*NET Program are meant to be exhaustive. The primary sources of data
are job incumbents and occupation experts. The SOC system is used as the basis for classifying
occupations. The use of questionnaires and rating scales reflects the most currently accepted
approach to job analysis. The scales used for the O*NET ratings are Importance, Level, and
Frequency. Each descriptor in the O*NET questionnaires may use one or more scales. For
example, the O*NET Skills descriptor Coordination is rated on both a 5-point Importance scale
and a 7-point Level scale. For the complete set of O*NET questionnaires, which include O*NET
descriptors, see Appendix A.
Exhibit 2 summarizes the number of descriptors and scales in the O*NET Data
Collection Program questionnaires. Descriptors are identified from O*NET Content Model
domains. Data are collected by means of 239 descriptors that include 400 scales (e.g.,
Importance, Level, Frequency). Currently, to collect ratings for the Abilities and Skills domains,
3
Justification
OMB Supporting Statement
Exhibit 2. O*NET Data Collection Program Questionnaires
O*NET Data Collection Program
Questionnaire
Number of
Descriptors
Skills
Knowledge
a
Work styles
Education and training
a
Number of Scales
per Descriptor
Total Number
of Scales
Data Source
35
2
70
Analysts
33
2
66
Job incumbents
16
1
16
Job incumbents
5
1
5
Job incumbents
Generalized work activities
41
2
82
Job incumbents
Work context
57
1
57
Job incumbents
Abilities
Tasks
b
Total (not including Tasks)
52
2
104
Varies
2
Varies
239
NA
400
Analysts
Job incumbents
NA
Notes: Occupation experts use the same questionnaires as job incumbents for those occupations whose data
collection is by the Occupation Expert Method. NA = not applicable.
a
The Knowledge Questionnaire packet also contains the Work Styles Questionnaire and the Education and Training
Questionnaire.
b
All job incumbents are asked to complete a Task Questionnaire in addition to the domain questionnaire.
trained occupational analysts review updated information (e.g., Tasks, Generalized Work
Activities) provided by job incumbents.3 No data collection is planned for the Workforce
Characteristics domain. Information for it is provided through links to the employment, wage,
and long-term projections databases produced by the U.S. Bureau of Labor Statistics (BLS), the
state employment security agencies, and other agencies.
Versions of the O*NET Database
The first version of the O*NET database released to the public was O*NET 98. O*NET
98 moved from the DOT’s more than 12,000 occupations to a more user-friendly 1,122
occupations based on the BLS Occupational Employment Statistics (OES) codes. The O*NET
98 database contained 306 descriptors and 684 scales. A review of O*NET 98–specific scales
and descriptors during the preparation for pretest data collection led to some consolidation and
refinement of descriptors and scales to reduce burden on the public and increase employee
response rate.4
The O*NET 98 database was first replaced with the O*NET 3.1 database and has been
updated 12 times as new data have been collected and analyzed. The current database, O*NET
version 16.0, contains the same descriptors used in O*NET 98; however, the occupations have
been restructured and coded to encompass the most detailed level of the 2010 SOC, with more
occupational specificity added as needed. Research is under way to identify additional new and
emerging occupations in high-growth industries. New occupations emerge because of changes in
3
4
For a discussion of the preferred data source, see Section A.1.2.
See Revision of O*NET data collection instruments, available at http://www.onetcenter.org/resData.html.
4
OMB Supporting Statement
Justification
technology, society, law, business practices, and markets. As these new and emerging
occupations are identified and their data are collected, they will be integrated into the O*NETSOC occupation classification and database.
O*NET 16.0 has a Web-based accessing application, O*NET OnLine, which is available
to the public at no cost at http://online.onetcenter.org/. An electronic version of the 16.0 database
can be downloaded at http://www.onetcenter.org/. The O*NET 16.0 database has been
restructured to incorporate improvements made to the O*NET data collection instruments and is
the structure currently being offered to developers.
Data in the O*NET database include the mean ratings on each of the items (or
descriptors) in the O*NET questionnaires. Ratings have been standardized to facilitate
interpretation and comparison. In addition to mean rating data on Level and Importance for
various questionnaire items, text information is also included on occupational definitions,
descriptor definitions, scale anchors, and task descriptions.
The Foundation for the O*NET Program
The DOT, first published by DOL in 1939, provided the occupational classification and
descriptions required for operation of the public employment service. Information for the DOT
was collected primarily through observation and interview by trained occupational analysts.
These collection techniques were applied to a small number of job incumbents from what can
best be characterized as a convenience sample. The DOT has not been maintained or updated
since 1991; many of its observations were conducted during the 1970s.
In the late 1980s the Employment and Training Administration (ETA) of DOL began a
review of the DOT program. The purpose was to address concerns about the cost and difficulty
of maintaining the DOT with existing methods and to address the need for additional and more
current information. In 1990, as part of the DOT review, the Secretary of Labor appointed the
Advisory Panel for the Dictionary of Occupational Titles. In response to its charge to advise on
improvements to the DOT, this panel called for development of a new database of occupational
information, a concept that evolved into the O*NET database. An important theme throughout
the panel’s recommendations was the development of a common language about jobs,
occupations, and skills. The panel’s complete report is presented in Appendix C.
A.1.2 The O*NET Data Collection Approach
The O*NET Data Collection Program is key to the continued effort to update the O*NET
database to reflect the most current occupational skills and attributes, with special emphasis on
high-growth sectors of the economy and new and emerging occupations. In the research leading
to the O*NET Data Collection Program, various sources and methods for collecting occupational
5
Justification
OMB Supporting Statement
information were examined, including collection of data from job incumbents and supervisors
and development of ratings by occupation experts and occupational analysts. On the basis of this
work, the O*NET team determined that the preferred source of data for most domains
(Generalized Work Activities; Work Context; Knowledge; Education and Training; and Work
Styles) is job incumbents. Other occupation experts, such as supervisors and trainers, may be
used where access to job incumbents proves difficult or where the sampling of business
establishments is inefficient.
By contrast, occupational analysts, who are provided with updated information from job
incumbents, are preferred for the Abilities domain, which tends to be more abstract than the
other domains. The Skills domain, whose variables are somewhat abstract, is a strong candidate
for either source of collection; it is now updated by occupational analysts. A study conducted in
2006 found no clear evidence that one source of raters provides more valid or accurate data than
the other for the Skills domain (Tsacoumis & Van Iddekinge, 2006).5 Consequently,
considerations of relative practicality, such as cost, informed the decision to proceed with analyst
ratings of both abilities and skills.
Previous studies comparing various sources of job analysis ratings suggest that
incumbents are able to provide information across a variety of descriptor domains (Fleishman &
Mumford, 1988; Peterson, Owens-Kurtz, Hoffman, Arabian, & Whetzel, 1990). In addition,
“large samples of knowledgeable job incumbents are available, which should contribute to the
reliability of the resulting descriptive system” (Peterson, Mumford, Levin, Green, & Waksberg,
1999, pp. 2–6). Furthermore, the world of work is constantly changing, and technological
advancements are occurring so rapidly that an efficient, effective way to remain current and
accurate is to obtain the information directly from those performing the work.
As part of a random sample, workers selected to participate in the O*NET Data
Collection Program are asked to rate the requirements of their own jobs as defined by the
O*NET questionnaire items. The responses are tabulated into statistics, such as mean ratings for
each scale. Collecting representative information from job incumbents presents many challenges;
among them is determining the best method for identifying a representative sample of job
incumbents in each occupation. The Advisory Panel for the Dictionary of Occupational Titles
had recommended using sampling techniques that would ensure the representativeness and
accuracy of the occupational data. Sampling allows an estimate of the population. By contrast,
no systematic sampling methods were used in the development of the original DOT. The O*NET
5
The goal of the study was to compare the psychometric quality of incumbent skills ratings with that of analyst
skills ratings across a large sample of O*NET-SOCs. Although some mean differences between incumbent and
analyst ratings were observed, the results yielded only minimal differences between the two systems of obtaining
skills information, supporting previous research.
6
OMB Supporting Statement
Justification
Program is concerned, in particular, with identifying sampling approaches that minimize burden
on employers and the public, achieving broad coverage of the workers in each occupation,
ensuring acceptable response rates, and supporting overall cost-efficiency.
Each sampling option has been evaluated in light of these criteria; each offers advantages
and disadvantages. Three types of sampling frames are available for identifying samples of
workers in each occupation: (1) lists of individual workers identified through professional and
trade associations, licensing agencies, and unions; (2) households; and (3) employer
establishments.
Identifying sampling frames of workers through professional and trade associations and
unions retains the advantage of lower response burden because contacts with a sample of
employers are replaced with contact with one or a few associations. Although it adds the cost of
soliciting and maintaining association cooperation, this procedure also removes the cost of
soliciting and maintaining employer cooperation. However, a major limitation of using special
sampling frames of professional and trade associations and unions is coverage: rarely does
association membership encompass a broad coverage of employment in the occupation. In
addition, the membership of many associations consists of people in multiple occupations, retired
individuals, and other interested parties. Moreover, few associations keep occupational
information on their membership, causing the identification of job incumbents in a specific
occupation to be problematic. Where coverage of employment in an occupation by association
membership is significant and members of the occupation in the association can be identified, a
special frame can sometimes be used to supplement the Establishment Method in a dual-frame
sample design.6
Persons in various occupations could also be identified and interviewed using a sample of
households. The primary disadvantage of this approach is cost. Because it is impossible to know
what occupations would be represented by the employed members of a household, many
households would have to be sampled and screened in order to identify and interview persons in
specific occupations. As an example, consider that there are fewer than 100,000 veterinarians in
the United States and about 61.3 million households. This means that more than 613 households
must be screened to find just one veterinarian. Some occupations are much rarer and would
require screening many households to locate and interview the required number for the O*NET
survey. The cost of this approach would be prohibitive given that other, more economical options
are available.
The Establishment Method provides the advantage of lower response burden and cost
when compared with a household survey, as well as good coverage for the large majority of
6
This dual-frame sample design is further discussed in Section B.1.1.
7
Justification
OMB Supporting Statement
occupations. Response burden and costs are lower for two primary reasons: (1) there are more
workers per employer than per household, so fewer contacts are required to identify workers; and
(2) employer contacts can be minimized by focusing on those most likely to employ workers in
each occupation for which the sample is required. Efficient sample design is possible because the
distribution of employment in an occupation is usually a function of the industry of the
employer. Employment by occupation by industry is measured by the federal-state OES program
national estimates provided by BLS.
The Establishment Method provides good coverage of wage and salary employment so
long as an acceptable employer sampling frame is available. Coverage of self-employment is
more difficult, although the sampling frame used in O*NET surveys includes many
establishments operated by self-employed workers. According to BLS (2011), coverage of
unpaid family workers in an employer sampling frame is minimal or zero, but the number of
workers involved is quite small (84,000 nationwide in 2010).
The Establishment Method, pretested in 1999 and 2000, remains the primary way to
update the O*NET database; most data are currently collected this way.7 Achieving high
response rates with the Establishment Method can be problematic, however, because it requires
cooperation at two levels: that of the employer and that of the sampled worker. To date, high
levels of cooperation nonetheless have been attained, and this method has proved successful.
Although the resulting response rates have been acceptable, the O*NET team continually works
to enhance response rates.8
An alternative method using occupation experts is employed to optimize the use of
burden hours and resources because some occupations are difficult to sample efficiently. This
situation occurs when it is difficult to locate industries or establishments with occupation
incumbents, employment is low, or employment data are not available, as is the case for many
new and emerging occupations. With the Occupation Expert (OE) Method, persons considered
experts in the target occupation are surveyed. These experts include supervisors and trainers, as
well as experienced job incumbents. The limitation of the OE Method is that locating experts can
be difficult. For some occupations, identifying a professional association proves difficult; in
other cases the association may lack membership information sufficient to identify experts for a
specific occupation.
Using the most appropriate sources of information (e.g., workers, occupation experts, and
analysts) and a multiple-method approach, the O*NET Data Collection Program efficiently
collects and yields high-quality occupational data.
7
8
For a description of the pretest, see Section B.4.1.
For a discussion of current and future efforts to improve response rates, see Section B.3.
8
OMB Supporting Statement
Justification
A.1.3 Summary of the O*NET Data Collection Process
The O*NET data collection process is broadly summarized here and detailed later in
separate sections of this Supporting Statement.
Sample Design
The O*NET Program sampling approaches are designed to create and update the O*NET
database in a highly cost-efficient and timely manner while maximizing the reliability of the
information in it. The primary method for collecting this information is the Establishment
Method, a survey of workers employed in a national probability sample of establishments.
Approximately 75% of occupations are completed by the Establishment Method. The method
uses a stratified two-stage design. At the first stage, a sample of businesses is selected from a
national database of more than 15 million establishments provided by Dun & Bradstreet (D&B).
The sample is selected with probability proportional to the expected number of employed
workers in the specific occupations being surveyed. At the second stage, a sample of workers is
selected in the occupations within the sampled businesses.
For selected occupations that are difficult to complete and additional observations are
required, a special frame, such as a professional of trade association membership list, is
sometimes used to supplement the D&B sample. The sample selection procedures vary across
associations, depending on the type of information available on association members. In general,
association lists are sampled with a single-stage, stratified simple random sampling approach.
Stratification by geographic location and occupation subspecialty is considered if it is
appropriate for the occupation.9
The OE Method is considered for use when the Establishment Method would likely be
problematic because of occupations with very low rates of employment, new and emerging
occupations lacking industry employment data, or occupations whose incumbents are in remote
locations that are difficult to access. The OE Method can be used only if the occupation is well
represented by one or more professional or trade associations that are willing and able to identify
experts in the target occupation. For this method, stratified samples of experts are selected from
lists of potential respondents. These potential experts are questioned to determine whether they
meet the specified criteria to serve as an occupation expert for their respective occupation.
Approximately 25% of the O*NET-SOC occupations are completed by the OE Method.10
Data Collection
Data collection operations are conducted at the contractor’s Operations Center in Raleigh,
North Carolina, and its Survey Support Department in Research Triangle Park, North Carolina.
9
For additional information about sampling with the Establishment Method, see Section B.1.2.
For additional information about sampling with the OE Method, see Section B.1.3.
10
9
Justification
OMB Supporting Statement
For the Establishment Method, the Operations Center’s Business Liaisons (BLs) contact sample
business establishments, secure the participation of a point of contact (POC), and work with the
POC to carry out data collection in the target occupations. The data are provided by randomly
selected employees in the occupations of interest. All within-establishment data collection is
coordinated by the POC; the BLs do not contact employees directly. After the POC agrees to
participate, informational materials and questionnaires are mailed to the POC, who distributes
the questionnaires to the sampled employees. Completed questionnaires are returned directly to
the survey contractor for processing. Respondents also have the option of completing the survey
online.
The Establishment Method works well for most occupations. Occasionally, however, the
standard protocol is supplemented with a special frame, such as a professional association
membership list, when additional observations are required to complete data collection for an
occupation. The primary difference with this approach is that the supplemental respondents are
sampled directly from the special frame and contacted directly by the BLs, without involvement
of a sampled establishment or a POC.
Survey support staff mail materials to POCs, job incumbents, and occupation experts, and
they receive and process completed questionnaires returned by respondents. Both the telephone
operations of the BLs and the mailing and questionnaire-receipt operations of the survey support
staff are supported by a Case Management System (CMS). Data-entry software supports the
keying and verification of incoming survey data.
Three domain questionnaires are used to collect data from sampled workers: Knowledge
(including Work Styles and Education and Training), Generalized Work Activities, and Work
Context. Each sampled worker is randomly assigned one of the three questionnaires. The
workers are also asked to provide basic demographic information and to complete a brief task
inventory for their specific occupation. Workers may complete the paper questionnaire and
return it by mail, or they may choose to complete the questionnaire online at the project Web
site. Questionnaires are available in Spanish for selected O*NET-SOC occupations. Data for two
domains, Abilities and Skills, are provided by trained analysts because of the more abstract
nature of the questions.11
Data Cleaning; Identification and Evaluation of Anomalous Cases
Data cleaning procedures eliminate completely blank questionnaires and insert consistent
analysis codes for legitimate skips, blank items, and invalid responses. Anomalous cases are
identified so respondents may be removed whose responses either suggest that they are not
working in the occupation of interest or are highly inconsistent with those of the others
11
For the list of questionnaires, number of items and scales, and data sources, see Exhibit 2.
10
OMB Supporting Statement
Justification
responding for the occupation. Unusable cases are identified according to prescribed eligibility
criteria, such as percentage of items completed. Cases with certain questionable characteristics
are flagged for further analysis. These flagged cases include cases with response patterns
deviating from those of other cases in the occupation and cases with write-in job titles that do not
appear to match the occupation. Responses judged invalid by expert reviewers are excluded from
the analysis file.12
Weighting and Estimation
Estimates generated from O*NET survey data are computed with sampling weights that
compensate for the unequal probabilities of selecting establishments, occupations within
establishments, and employees within each selected occupation. In addition, these base weights
are adjusted to further compensate for multiple subwaves of sampling, sample adjustment,
population under- and overcoverage caused by frame imperfections, and nonresponse at both the
establishment and the employee levels.
These weight adjustments can lead to weights that are very large or very small compared
with the weights for other sample units. Such weight variability may increase the standard error
estimates. When the variation in the weights is large, it is desirable to trim the weights to reduce
the variation. For the O*NET estimates, the weighting process involves a weight trimming
procedure in which extremely large or small weights are truncated to fall within a specified
range. Although trimming weights can introduce bias in the estimates, the variance reduction it
achieves usually offsets the potential bias, resulting in estimates with smaller net mean squared
errors.
Based on a pooled sample of all completed waves, final estimates are produced.
Estimates are computed by summing the weighted observations and dividing by the sum of the
weights. Standard errors are estimated with the first-order Taylor series approximation of
deviations of estimates from their expected values. These design-based variance estimates are
computed with SUDAAN® software (RTI International, 2004). These estimates properly account
for the combined effects of clustering, stratification, and unequal weighting—all of which are
present in the O*NET data. In addition, estimates with questionable precision are flagged
“recommended for suppression” in the O*NET database.13
Analysis
Nonresponse is analyzed at multiple levels. Establishment nonresponse in the O*NET
Data Collection Program can occur at the verification, screening, recruiting, or sampling stages
12
13
For a description of both processes, see Section A.16.1.
For more information on the calculation of weights and variance estimates, see Section B.1.1. For information on
the suppression of estimates, see Section A.16.1.
11
Justification
OMB Supporting Statement
of selection. Employee-level nonresponse occurs when a selected employee fails to complete and
return a questionnaire. Because these same characteristics of establishments and employees were
used to adjust the analysis weights, compensation is made to further reduce the size and impact
of any nonresponse bias. In addition, item-level nonresponse is assessed, and item response rates
are high—generally higher than 90% for Likert scale items, with most exceeding 95%.
For each O*NET-SOC occupation, the degree of interrater reliability (the covariation
among ratings) and the level of interrater agreement (the absolute difference among ratings) is
calculated annually. The results of the analyses are used to examine the potential sources of
variability across respondents in a specific occupation. As part of a continuous improvement
process, these results also inform an evaluation of the O*NET-SOC occupational taxonomy,
Content Model descriptors, and scales.14
A.1.4 Summary of Response Rate Experience to Date
Establishment Method
Data collection for the O*NET Data Collection Program began in June 2001 and has
been in continuous operation since then. Exhibit 3 shows our cumulative response rate
experience as of December 31, 2011.
Exhibit 3. Establishment Method Data Collection Results
Sampled establishments
203,266
Eligible establishments
163,834
Participating establishments
124,598
Establishment response rate (eligible establishments/participating establishments)
76.1%
Eligible employees
248,507
Participating employees
161,505
Employee response rate (eligible employees/participating employees)
65.0%
As indicated, more than 124,000 establishments and 161,000 employees have responded
to the survey request, resulting in an establishment response rate of 76% and an employee
response rate of 65%. Although it is difficult to make response rate comparisons between the
O*NET Data Collection Program and other establishment surveys, these rates compare favorably
with those of other establishment surveys.
14
For a description of analysis processes, see Sections A.16.1 and A.16.2.
12
OMB Supporting Statement
Justification
Some of the design characteristics that make the O*NET Program unique are as follows:
Voluntary rather than mandatory participation. Experiments conducted by the U.S.
Bureau of the Census suggest that simply informing respondents that their response is
mandatory adds about 20 percentage points to overall response (Tulp, Hoy, Kusch, &
Cole, 1991; Worden & Hoy, 1992). Because the survey is voluntary, O*NET
response rates would be expected to be 10 to 20 percentage points lower than those
for the average federally mandated survey.
No direct, personal contact with the survey respondents by the survey organization
conducting the data collection. The O*NET Program requirement of respondent
anonymity means that participation at the employee level relies exclusively on the
interactions between the POC and the employee. The survey organization is not
allowed to speak to the employee to respond to questions, motivate responses, or
follow up on noncompliance. In their review of establishment mail survey response
rates, Paxon, Dillman, and Tarnai (1995) found that establishment surveys featuring
anonymous mailings typically have lower response rates (by as many as 30
percentage points) than surveys featuring direct, personal contact with the
respondents.
Participation required at three stages of response—establishment level, point of
contact level, and employee level. The typical establishment survey requires
participation at only one or two levels: the establishment level and, in some cases, the
POC level. By contrast, for the O*NET Program three often distinct entities must
agree to participate: the establishment administration, the POC identified in the
screening interview, and the employee who is asked to complete the questionnaire.
Because very few surveys incorporate such a design, survey methods literature is
essentially devoid of examples on which to base a reasonable response rate
expectation for the O*NET Data Collection Program. However, it is possible to
compare O*NET response rates at each stage with other establishment surveys that
incorporate these stages either separately or in combination. For example, the O*NET
establishment-level response rate can be compared with other mail establishment
surveys having only one response stage at the establishment level. In addition, the
O*NET employee-level response rate can be compared with the response rate of other
establishments’ self-conducted employee surveys.
A high level of nonresponse is a threat to the accuracy of the estimates. For this reason,
the O*NET data collection protocol (described in Sections B.2 and 0B.3) was designed to
achieve the highest response rates possible within respondent burden and survey cost constraints.
Paxon and colleagues (1995) analyzed the response rates for 46 surveys conducted by both
governmental and nongovernmental organizations. Among the surveys in their study, 26 were
conducted by the Social and Economic Sciences Research Center (SESRC) at Washington State
University, and 20 were conducted by the U.S. Bureau of the Census. The SESRC, directed by
Dr. Donald Dillman, is well known for its development of the Total Design Method approach to
mail surveys (Dillman, 1978, 2000) and its high response rates in implementing this method.
Furthermore, the 20 U.S. Census Bureau surveys include 12 well-established, ongoing
13
Justification
OMB Supporting Statement
mandatory surveys. The average response rate for all 46 surveys is 63%, but, if only voluntary
surveys are considered, the average response rate drops to 55%. These results suggest that, for
this type of survey, the O*NET establishment-level response rate of 76.1% exceeds expectations.
In a more recent review of four major voluntary establishment-based surveys conducted
by BLS, initial unweighted data collection response rates (defined as the percentage of sampled
establishments that agreed to provide any of the requested data), ranged from 66% to 87%
(Petroni, Sigman, Willimack, Cohen, & Tucker, 2004). Additionally, the Census Bureau’s
Monthly Retail Sales Survey and Monthly Wholesale Survey, both of which are voluntary mail
surveys with a telephone follow-up, show cooperation rates ranging between 66% and 80%
(W. Davie, personal communication, March 23, 2011). The O*NET establishment-level response
rate of 76% falls well within the midrange of the Census Bureau’s response rates range.
Comparisons of the O*NET employee response rate can be made with surveys that
directly sample employees within an establishment. Because federally sponsored surveys of
employees within organizations are rare, the literature on their response rates is sparse. Most
surveys of this type are employee satisfaction surveys. For example, one well-documented
government-sponsored survey of employees is the Public Service Employee Survey,
administered to about 258,000 employees of the federal Public Service of Canada and conducted
by Statistics Canada in 2008 (Treasury Board of Canada Secretariat, 2009, May). A
questionnaire was delivered to each employee by a government agent who personally requested
that the employee complete the questionnaire and return it by mail. Multiple follow-ups of
nonrespondents were made by e-mail and interoffice mail to maximize the response rate. No
incentive was used, however; because all sample members were also employees of the
organization conducting the survey and could fill out the survey on government time, the use of
an incentive was thought to be unnecessary. The final overall response rate for the survey was
66%.
The Office of Personnel Management (OPM) developed the Organizational Assessment
Survey (OAS) and has encouraged all federal agencies to survey their employees in order to
evaluate organizational performance, benchmark best practices, and align performance with
important and measurable outcomes. The experience OPM has had in implementing these
surveys in numerous federal agencies provides some evidence of response rates for employee
surveys conducted by the U.S. federal government. The OAS design closely resembles the
Canadian Public Service Employee Survey design. The surveys are self-administered and are
conducted by each agency for its own employees. Furthermore, like the Public Service Employee
Survey, the OAS request to participate is personalized and made directly to the employee by his
or her employer. The features of the design offer a significant advantage over the O*NET survey
design, as previously noted.
14
OMB Supporting Statement
Justification
Although the results of the OAS surveys are not publicly available, an official at OPM
was able to provide some general information regarding OAS response rates (C. Simons,
personal communication, March 21, 2002). According to OPM, response rates for OAS surveys
vary considerably by agency, from 30% to 80%. However, the average response across all
agencies is approximately 57%. The Employee Viewpoint Survey (L. Pollack, personal
communication, March 23, 2011), a biannual census administered to hundreds of thousands of
full-time federal employees across U.S. government agencies, yielded response rates of 54% in
2004, 57% in 2006, 51% in 2008, and 52% in 2010 (U.S. Office of Personnel Management,
2010) Additionally, Roth and BeVier (1998), who conducted a meta-analysis of 173 surveys in
the field of human resource management and organization behavior, found that surveys
implementing many of the response-rate-enhancing features of the O*NET survey had response
rates in the range of 23% to 78%, with a median of 51%. These results suggest that the O*NET
employee response rate of 64.9% exceeds expectations for mail surveys of employees within
their organizations.
Although results from the survey methods literature and from other federal surveys in the
United States and in Canada suggest that a 76% establishment response rate and a 65% employee
response rate exceed expectations for federal surveys that use similar data collection approaches,
methods for further improving response rates will continue to be explored, and response rates are
fully expected to continue to increase as the O*NET Data Collection Program progresses.15
Occupation Expert Method
The OE Method is a much smaller but still important component of the O*NET Data
Collection Program protocol. Exhibit 4 shows our response rate experience with this method as
of December 31, 2011.
Exhibit 4. Occupation Expert Method Data Collection Results
Sampled occupation experts
10,081
Eligible occupation experts
7,580
Participating occupation experts
6,050
Occupation expert response rate (eligible occupation experts/participating occupation experts)
79.8%
As indicated, 6,050 of 7,580 eligible OEs have participated, for a response rate of 80%.
This response rate is higher than the employee response rate for the Establishment Method and
may be due to factors such as personalized direct contact, greater total incentive, and generally
15
See Section B.3 for a description of methods used to maximize response rates and evaluate the potential for
nonresponse bias.
15
Justification
OMB Supporting Statement
higher education levels of the eligible population. Some factors may counteract these positive
effects, including greater respondent burden and task complexity, as well as potentially less
available time for occupations than for the average employee.
A.1.5 Statutory and Regulatory Information
Although the O*NET name is not referenced specifically in statutes, it is cited at least six
times in the U.S. Code of Federal Regulations (C.F.R.), and the DOT—which O*NET largely
replaced—is cited 27 times in the C.F.R. (the DOT was formerly cited in the statutes, but there
are no current statutory references to it). Furthermore, O*NET information is the foundational
common language for fulfilling the statutory and regulatory responsibilities discussed in this
section.
Section 309 of the Workforce Investment Act ("Workforce Investment Act, Pub. L. 105220, 112 Stat. 936 (1998),") requires the Secretary of Labor to oversee the “development,
maintenance, and continuous improvement of a nationwide employment statistics system,”
which shall include, among other components, “skill trends by occupation and industry.” The
O*NET program is the primary means for collecting skills information across all occupations.
Updating the entire O*NET database is a critical component of the nationwide labor market
information system to support employer, workforce, and education information needs.
The WIA contains numerous references that identify the need for information on the skill
requirements of jobs. For example, Section 154 requires local determinations of the “skills and
education that are necessary to obtain the employment opportunities” in the local area, and
Section 134 requires provision of “information on job skills necessary” for obtaining jobs listed
for the local area.
Other WIA references address the need for information on the skills of individuals. For
example, Section 134 provides for assessment services to identify “the skill levels and service
needs of adults and dislocated workers.” Section 129 allows funds to be used to provide youth
with an assessment “which shall include a review of basic skills, occupational skills, prior work
experience, employability, interests, aptitudes (including interests and aptitudes for
nontraditional jobs), supportive service needs, and development needs of such participant….” In
section 136, the “attainment of basic skills and, as appropriate, work readiness or occupational
skills” may be included as performance indicators for youth programs. The O*NET Career
Exploration Tools, including the O*NET Interest Profiler and O*NET Work Importance
Locator, are such assessment tools, designed specifically to relate a person’s interests and work
values to the information on education and skill requirements for occupations in the O*NET
database.
16
OMB Supporting Statement
Justification
Section 508 of the Rehabilitation Act of 1973, as amended, specifies that persons with
disabilities shall have access to and use of the same information that persons without disabilities
have. To comply with this section, the O*NET Data Collection Program designed its online
application, O*NET OnLine, to ensure that the data and information are accessible to the widest
possible audience, including persons with disabilities. O*NET OnLine also provides links to
several accommodation and disability resources on the Internet. Furthermore, the O*NET
program Web site, DOL ETA’s O*NET Web site, and O*NET OnLine have all been “Bobby
Approved,” indicating that the Center for Applied Special Technology has deemed them
accessible to persons with disabilities.
The Secretary of Labor’s Strategic Plan includes O*NET within Strategic Goal 1: Prepare
workers for good jobs and ensure fair compensation. O*NET also addresses the following
Outcome Goals under Strategic Goal 1.
Outcome Goal 1.2: Assure skills and knowledge that prepare workers to succeed in a
knowledge-based economy, including in high growth and emerging industry sectors
like “green” jobs.
Outcome Goal 1.3: Help workers who are in low-wage jobs or out of the labor market
find a path into middle-class jobs.
Outcome Goal 1.4: Help middle-class families remain in the middle class.
The purpose of the proposed data collection effort is to keep O*NET skills information
current, which is key to supporting the needs of customers through the wide variety of publicand private-sector products that depend on the availability, quality, and currency of O*NET
information. O*NET supplies the common language on skills, knowledge, abilities, and other
work requirements and characteristics.
Finally, the predecessor to the O*NET database, the DOT, is frequently cited as a source
of occupational information in support of Federal programs. The 27 citations of the DOT in the
C.F.R. include references to determining disability, administering DOL programs, and
administering immigration, civil rights, and labor standards law. DOL officials responsible for
the O*NET Program work with various Federal users of the DOT, some of whom have made the
transition either in regulatory changes or in practices and procedures. These include State
Department officials responsible for visas, the U.S. Office of Apprenticeship, and the Office of
Foreign Labor Certification. The specific C.F.R. citations appear in Exhibit 5.
17
Justification
OMB Supporting Statement
Exhibit 5. O*NET Citations in Code of Federal Regulations
1. 20 C.F.R. 651.10 - Definitions of terms used in parts 651-658. [PDF 62 KB] [p. 432]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Part 651: GENERAL PROVISIONS
GOVERNING THE FEDERAL-STATE EMPLOYMENT SERVICE SYSTEM. Friday, April 1, 2011.
... Occupational Information Network (O*NET) means the online reference database...work styles, and work
values. O*NET-SOC means Standard Occupational...calculating and disseminating data. DOL uses O*NET-SOC
titles and codes for the... More Information
Historical editions have been hidden from results. Show all editions.
2. 20 C.F.R. 656.3 - Definitions, for purposes of this part, of terms used in this part. [PDF 59 KB] [p. 752]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart A: Purpose and Scope of
Part 656. Friday, April 1, 2011.
...and Nationality Act, as amended. O*NET means the system developed by the...levels associated with
occupations. O*NET is based on the Standard Occupational...system. Further information about O*NET can be
found at... More Information
Historical editions have been hidden from results. Show all editions.
3. 20 C.F.R. 655.1308 - Offered wage rate. [PDF 52 KB] [p. 738]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart N: Labor Certification
Process for Temporary Agricultural Employment in the United States (H-2A Workers). Friday, April 1, 2011.
...experience that are generally required as described in the O*NET Job Zones. (3) Level III wage rates are
assigned...educational degrees that are at the higher ranges indicated in the O*NET Job Zones would be an
indicator that a Level III... More Information
Historical editions have been hidden from results. Show all editions.
4. 20 C.F.R. 656.17 - Basic labor certification process. [PDF 67 KB] [p. 763]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart C: Labor Certification
Process. Friday, April 1, 2011.
...the occupation and must not exceed the Specific Vocational Preparation level assigned to the occupation as
shown in the O*NET Job Zones. To establish a business necessity, an employer must demonstrate the job duties
and requirements bear a... More Information
Historical editions have been hidden from results. Show all editions.
5. 20 C.F.R. 653.103 - MSFW job applications. [PDF 54 KB] [p. 444]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart B: Services for Migrant and
Seasonal Farmworkers (MSFWs). Friday, April 1, 2011.
...ES-511). Additional Occupational Informational Network (O*NET) codes or keywords shall be assigned, where
appropriate...does not require completion of secondary cards, additional O*NET-SOC codes shall be noted on
the primary application... More Information
Historical editions have been hidden from results. Show all editions.
6. 22 C.F.R. 42.33 - Diversity immigrants. [PDF 62 KB] [p. 208]
Code of Federal Regulations (annual edition). Title 22: Foreign Relations. Subpart D: Immigrants Subject to
Numerical Limitations. Friday, April 1, 2011.
...and Diversity Visa Programs occurring in subsequent fiscal years, consular officers must use the Department of
Labor's O*Net On Line to determine qualifying work experience. (4) Limitation on number of petitions per year.
No more... More Information
Historical editions have been hidden from results. Show all editions.
continued
18
OMB Supporting Statement
Justification
Exhibit 5. O*NET Citations in the Code of Federal Regulations (continued)
7.
20 C.F.R. 655.730 - What is the process for filing a labor condition application? [PDF 55 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart H: Labor Condition
Applications and Requirements for Employers Seeking To Employ Nonimmigrants on H-1b Visas in Specialty
Occupations and as Fashion Models, and Requirements for Employers Seeking To Employ Nonimmigrants on
H-1b1 and E-3 Visas in Specialty Occupations. Friday, April 1, 2011.
...occupational classification for which the LCA is being submitted and shall state: (i) The occupation, by
Dictionary of Occupational Titles (DOT) Three-Digit Occupational Groups code and by the employer's own
title for the job; (ii)... More Information
Historical editions have been hidden from results. Show all editions.
8.
20 C.F.R. 655.940 - Employer attestations. [PDF 65 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart J: Attestations by
Employers Using F-1 Students in Off-Campus Work. Friday, April 1, 2011.
...which the attestation is provided, state the occupational division in which the position is located, by Dictionary
of Occupational Titles (DOT) Two-Digit Occupational Divisions code, and shall state the rate(s) of pay for the...
More Information
Historical editions have been hidden from results. Show all editions.
9.
20 C.F.R. 627.240 - On-the-job training. [PDF 55 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart B: Program Requirements.
Friday, April 1, 2011.
...average training time, consideration should be given to recognized reference materials, such as the
“Dictionary of Occupational Titles”(DOT)andemployertrainingplans.Suchmaterialsneednotbelimitedto
the DOT, however.... More Information
Historical editions have been hidden from results. Show all editions.
10. 20 C.F.R. 655.10 - Determination of prevailing wage for temporary labor certification purposes. [PDF 55 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart A: Labor Certification
Process and Enforcement of Attestations for Temporary Employment in Occupations Other Than Agriculture or
Registered Nursing in the United States (H-2B Workers). Friday, April 1, 2011.
...DBA, or SCA wage available for the job opportunity; (B) The job opportunity was not listed in the Dictionary
of Occupational Titles (DOT) and is not listed in the Standard Occupational Classification (SOC) system, or if
the job... More Information
Historical editions have been hidden from results. Show all editions.
11. 29 C.F.R. 553.103 - “Sametypeofservices”defined. [PDF 53 KB]
Code of Federal Regulations (annual edition). Title 29: Labor. Subpart B: Volunteers. Thursday, July 1, 2010.
...the duties and other factors contained in the definitions of the 3-digit categories of occupations in the
Dictionary of Occupational Titles indeterminingwhetherthevolunteeractivitiesconstitutethe“sametypeof
services”asthe...More Information
Historical editions have been hidden from results. Show all editions.
12. 41 C.F.R. 60-3.15 - Documentation of impact and validity evidence. [PDF 69 KB]
Code of Federal Regulations (annual edition). Title 41: Public Contracts and Property Management. Subjgrp:
Documentation of Impact and Validity Evidence. Thursday, July 1, 2010.
...s) from U.S. Employment Service's Dictionary of Occupational Titles. (5) Criterion measures. The
bases...the United States Employment Service's dictionary of occupational titles. (6) Selection procedure.
The... More Information
Historical editions have been hidden from results. Show all editions.
continued
19
Justification
OMB Supporting Statement
Exhibit 5. O*NET Citations in the Code of Federal Regulations (continued)
13. 29 C.F.R. 553.30 - Occasional or sporadic employment-section 7(p)(2). [PDF 53 KB]
Code of Federal Regulations (annual edition). Title 29: Labor. Subjgrp: Other Exemptions. Thursday, July 1,
2010.
...the duties and other factors contained in the definitions of the 3-digit categories of occupations in the
Dictionary of Occupational Titles (except in the case of public safety employees as discussed below in
section (3)), as well as... More Information
Historical editions have been hidden from results. Show all editions.
14. 28 C.F.R. 50.14 - Guidelines on employee selection procedures. [PDF 129 KB]
Code of Federal Regulations (annual edition). Title 28: Judicial Administration. Part 50: STATEMENTS OF
POLICY. Thursday, July 1, 2010.
...s) from U.S. Employment Service's Dictionary of Occupational Titles. (5) Criterion measures. The
bases...the United States Employment Service's dictionary of occupational titles. (6) Selection procedure.
The... More Information
Historical editions have been hidden from results. Show all editions.
15. 34 C.F.R. 600.2 - Definitions. [PDF 58 KB]
Code of Federal Regulations (annual edition). Title 34: Education. Subpart A: General. Thursday, July 1, 2010.
... An occupation that is— (1)Listedinan“occupationaldivision”ofthelatest edition of the Dictionary of
Occupational Titles, published by the U.S. Department of Labor; or (2) Determined by the Secretary in... More
Information
Historical editions have been hidden from results. Show all editions.
16. 29 C.F.R. 1607.15 - Documentation of impact and validity evidence. [PDF 70 KB]
Code of Federal Regulations (annual edition). Title 29: Labor. Documentation of Impact and Validity Evidence.
Thursday, July 1, 2010.
...s) from U.S. Employment Service's Dictionary of Occupational Titles. (5) Criterion measures. The
bases...the United States Employment Service's dictionary of occupational titles. (6) Selection procedure.
The... More Information
Historical editions have been hidden from results. Show all editions.
17. 20 C.F.R. 404.1567 - Physical exertion requirements. [PDF 53 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Vocational Considerations.
Thursday, April 1, 2010.
...sedentary, light, medium, heavy, and very heavy. These terms have the same meaning as they have in the
Dictionary of Occupational Titles, published by the Department of Labor. In making disability determinations
under this subpart, we... More Information
Historical editions have been hidden from results. Show all editions.
18. 20 C.F.R. 220.131 - Work which exists in the national economy. [PDF 51 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart K: Vocational
Considerations. Thursday, April 1, 2010.
...Board determines that unskilled, sedentary, light and medium jobs exist in the national economy: (1)
Dictionary of Occupational Titles, published by the Department of Labor. (2) County Business Patterns,
published by the... More Information
Historical editions have been hidden from results. Show all editions.
continued
20
OMB Supporting Statement
Justification
Exhibit 5. O*NET Citations in the Code of Federal Regulations (continued)
19. 20 C.F.R. 220.134 - Medical-vocational guidelines in appendix 2 of this part. [PDF 51 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart K: Vocational
Considerations. Thursday, April 1, 2010.
...Considerations § 220.134 Medical-vocational guidelines in appendix 2 of this part. (a) The Dictionary of
Occupational Titles includes information about jobs (classified by their exertional and skill requirements) that
exist in... More Information
Historical editions have been hidden from results. Show all editions.
20. 20 C.F.R. 220.135 - Exertional and nonexertional limitations. [PDF 51 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart K: Vocational
Considerations. Thursday, April 1, 2010.
...Board uses the classification of jobs by exertional levels (strength demands) which is contained in the
Dictionary of Occupational Titles published by the Department of Labor, to determine the exertional
requirements of work which exists... More Information
Historical editions have been hidden from results. Show all editions.
21. 20 C.F.R. 404.1569a - Exertional and nonexertional limitations. [PDF 56 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Vocational Considerations.
Thursday, April 1, 2010.
...how we use the classification of jobs by exertional levels (strength demands) which is contained in the
Dictionary of Occupational Titles published by the Department of Labor, to determine the exertional
requirements of work which exists... More Information
Historical editions have been hidden from results. Show all editions.
22. 20 C.F.R. 220.13 - Establishment of permanent disability for work in regular railroad occupation. [PDF 54 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart C: Disability Under the
Railroad Retirement Act for Work in an Employee's Regular Railroad Occupation. Thursday, April 1, 2010.
...employer on the appropriate form set forth in appendix 3 of this part, and consult other sources such as the
Dictionary of Occupational Titles and the job descriptions of occupations found in the Occupational Disability
Claims Manual, as provided... More Information
Historical editions have been hidden from results. Show all editions.
23. 20 C.F.R. Appendix 2 to Part 220 - Medical-Vocational Guidelines [PDF 63 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Part 220: DETERMINING
DISABILITY. Thursday, April 1, 2010.
...variousfunctionallevels(sedentary,light,medium,heavy,andveryheavy)assupportedbythe“Dictionary of
Occupational Titles”andthe“OccupationalOutlookHandbook,”publishedbytheDepartmentofLabor;the
“CountyBusiness...More Information
Historical editions have been hidden from results. Show all editions.
24. 20 C.F.R. 404.1566 - Work which exists in the national economy. [PDF 53 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Vocational Considerations.
Thursday, April 1, 2010.
...available from various governmental and other publications. For example, we will take notice of— (1)
Dictionary of Occupational Titles, published by the Department of Labor; (2) County Business Patterns,
published by the... More Information
Historical editions have been hidden from results. Show all editions.
continued
21
Justification
OMB Supporting Statement
Exhibit 5. O*NET Citations in the Code of Federal Regulations (continued)
25. 20 C.F.R. 404.1569 - Listing of Medical-Vocational Guidelines in appendix 2. [PDF 50 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Vocational Considerations.
Thursday, April 1, 2010.
...Vocational Considerations § 404.1569 Listing of Medical-Vocational Guidelines in appendix 2. The Dictionary
of Occupational Titles includes information about jobs (classified by their exertional and skill requirements)
that... More Information
Historical editions have been hidden from results. Show all editions.
26. 20 C.F.R. Appendix 2 to Subpart P of... - Medical-Vocational Guidelines [PDF 79 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart P: Determining Disability
and Blindness. Thursday, April 1, 2010.
...variousfunctionallevels(sedentary,light,medium,heavy,andveryheavy)assupportedbythe“Dictionary of
Occupational Titles”andthe“OccupationalOutlookHandbook,”publishedbytheDepartmentofLabor;the
“CountyBusiness...More Information
Historical editions have been hidden from results. Show all editions.
27. 20 C.F.R. 416.969 - Listing of Medical-Vocational Guidelines in appendix 2 of subpart P of part 404 of this
chapter. [PDF 50 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Vocational Considerations.
Thursday, April 1, 2010.
...Listing of Medical-Vocational Guidelines in appendix 2 of subpart P of part 404 of this chapter. The Dictionary
of Occupational Titles includes information about jobs (classified by their exertional and skill requirements)
that... More Information
Historical editions have been hidden from results. Show all editions.
28. 20 C.F.R. 416.960 - When we will consider your vocational background. [PDF 53 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Vocational Considerations.
Thursday, April 1, 2010.
...may use the services of vocational experts or vocational specialists, or other resources, such as the
“Dictionary of Occupational Titles”anditscompanionvolumesandsupplements, published by the
Department of Labor, to obtain... More Information
Historical editions have been hidden from results. Show all editions.
29. 20 C.F.R. 416.967 - Physical exertion requirements. [PDF 53 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Vocational Considerations.
Thursday, April 1, 2010.
...sedentary, light, medium, heavy, and very heavy. These terms have the same meaning as they have in the
Dictionary of Occupational Titles, published by the Department of Labor. In making disability determinations
under this subpart, we... More Information
Historical editions have been hidden from results. Show all editions.
30. 20 C.F.R. 416.969a - Exertional and nonexertional limitations. [PDF 53 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Vocational Considerations.
Thursday, April 1, 2010.
...how we use the classification of jobs by exertional levels (strength demands) which is contained in the
Dictionary of Occupational Titles published by the Department of Labor, to determine the exertional
requirements of work which exists... More Information
Historical editions have been hidden from results. Show all editions.
continued
22
OMB Supporting Statement
Justification
Exhibit 5. O*NET Citations in the Code of Federal Regulations (continued)
31. 20 C.F.R. 404.1560 - When we will consider your vocational background. [PDF 53 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Vocational Considerations.
Thursday, April 1, 2010.
...may use the services of vocational experts or vocational specialists, or other resources, such as the
“Dictionary of Occupational Titles”anditscompanionvolumesandsupplements, published by the
Department of Labor, to obtain... More Information
Historical editions have been hidden from results. Show all editions.
32. 20 C.F.R. 416.966 - Work which exists in the national economy. [PDF 53 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Vocational Considerations.
Thursday, April 1, 2010.
...available from various governmental and other publications. For example, we will take notice of— (1)
Dictionary of Occupational Titles, published by the Department of Labor; (2) County Business Patterns,
published by the... More Information
Historical editions have been hidden from results. Show all editions.
33. 20 C.F.R. 220.132 - Physical exertion requirements. [PDF 51 KB]
Code of Federal Regulations (annual edition). Title 20: Employees' Benefits. Subpart K: Vocational
Considerations. Thursday, April 1, 2010.
...medium”,“heavy”,and“veryheavy.”ThesetermshavethesamemeaningastheyhaveintheDictionary of
Occupational Titles, published by the Department of Labor. In making disability determinations the Board uses
the... More Information
Historical editions have been hidden from results. Show all editions.
The search for O*NET and the DOT in the most recent editions of the C.F.R. was performed on June 20, 2011,
using the U.S. Government Printing Office’sFederalDigitalSystematwww.gpo.gov/fdsys/search/home.action. Six
O*NET citations (the first six rows of the table) were identified in the C.F.R., along with 27 DOT citations (the
remaining rows).
A.1.6 Federal Register Notice
The Federal Register notice asking for public comment before submission of this request
to OMB was published on November 29, 2011 (vol. 76, p. 73680). Comments received and our
response are described in Section A.8.
A.2
Uses, Products, and Services Based on the O*NET Program
The O*NET Program provides essential tools and services for numerous critical federal
and state workforce investment functions. These tools also serve job seekers, businesses, and
educational institutions across the country. The common language used in O*NET occupational
and skill descriptions facilitates communication among the various user groups. This facilitation
contributes to our nation’s talent development and promotes U.S. competitiveness in the global
21st-century economy.
Users of the O*NET database can access and use the information in various ways. The
data are available to the public free of charge through the O*NET OnLine Web application, or
the database may be downloaded for use by developers who provide applications targeted to
23
Justification
OMB Supporting Statement
specific communities or audiences. The O*NET database is now the foundation for many
programs serving the American workforce, providing information to build transferable skills and
skills gap analyses, competency profiles and to facilitate cross-functional team building. These
examples are just a few of the ways the O*NET Program supports the activities critical to
maintaining a mobile workforce responsive to changing regional and national economic needs.
A.2.1 The O*NET Database, O*NET OnLine, My Next Move, O*NET Career Tools,
O*NET Training Academy, and O*NET Code Connector
The O*NET Database
With the July 2011 version of the database, 874 occupations have been comprehensively
updated through the O*NET Data Collection Program. Three hundred fifty-nine of these
occupations have undergone a second, more recent update. The O*NET database currently
provides
detailed occupational and skill information for more than 874 occupations;
information on standardized descriptors of skills, abilities, interests, knowledge, work
values, education, training, work context, and work styles; and
occupational coding based on the 2010 SOC.
Since September 2007 there have been four database updates as listed in Exhibit 6.
Exhibit 6. Database Updates
Update
Number of Occupations Updated
June 2008
108
June 2009
117
June 2010
120
July 2011
107
Recent updates of the O*NET database include identification of and information about
“green” and new and emerging occupations.16 Future database updates will significantly increase
coverage of these occupations.
O*NET OnLine
The O*NET database is provided free of charge to the public through O*NET OnLine, a
Web-based application at http://www.onetonline.org/. O*NET OnLine offers users multiple
ways to search for O*NET occupations and related data. O*NET OnLine includes
16
Green occupations will likely change as a result of the green economy. Green economy activities and technologies
are increasing the demand for occupations, shaping the work and worker requirements needed for occupational
performance, or generating new and emerging occupations. (See Dierdorff et al. [2009].)
24
OMB Supporting Statement
Justification
regularly updated occupational information;
tiered search algorithms that maximize successful results of searches by keyword
(title), occupational code, or partial code;
a crosswalk search function that enables users to easily identify O*NET equivalents
of other occupational systems, such as the Military Occupational Classification or the
Classification of Instructional Programs;
the ability to browse by O*NET Descriptor (this search enables users to make crossoccupational comparisons by viewing an occupation’s rank order on the basis of a
selected knowledge, skill, ability, work activity, interest, or work value variable);
the ability to browse groups of similar occupations, including bright outlook17; career
clusters; green economy sectors; industry; or science, technology, engineering, and
mathematics (STEM) disciplines;
a sample of reported job titles for each occupational report, providing the user with a
broader understanding of the O*NET-SOC;
report display options (aside from the default Summary Report, users can choose to
view a comprehensive Details Report or to build a Custom Report); and
inclusion of wages and employment trends information (derived from BLS data) and
links to Web sites of professional associations, giving the user access to additional
information on related specialties, industries, and education and training resources.
My Next Move
My Next Move (http://www.mynextmove.org/) is a new, easy-to-use, easy-to-read Webbased interactive tool for new job seekers, students, and other career explorers to learn more
about their career options. Users can explore more than 900 different careers and see important
information about them, including skills, tasks, salaries, and employment outlook. Users can also
look at related apprenticeships and training and search actual job openings. Users can find
careers by searching keywords; by browsing industries; or by using the O*NET Interest Profiler,
a tool that offers customized career suggestions based on a person’s interests and level of work
experience.
Career reports in My Next Move feature the most important knowledge, skills, and
abilities needed to perform the work, explained in language that’s easy to understand. Outlook
and education sections let users find salary info, job postings, and training opportunities. The
visual design enables users to identify a career’s key points or explore a career in depth.
Visitors interested in specific careers can start exploring quickly with an intuitive
keyword search; more than 900 career options are only a few keystrokes away. Users looking for
a broader range of opportunities can explore more than a dozen different industries; each features
17
Bright outlook occupations are new and emerging occupations or those expected to grow rapidly in the next
several years or to have large numbers of job openings.
25
Justification
OMB Supporting Statement
a range of careers to choose from, including those in the green economy and with a bright
outlook for job opportunities.
Also at the site is a Web-based version of the popular O*NET Interest Profiler, a tool
designed to assess an individual’s vocational interests. The Web-based version of the tool
features 60 items that, along with information about the user’s education and work experience,
guide users to careers they may enjoy. Find more information about the O*NET Interest
Profiler–Short Form at http://www.onetcenter.org/IPSF.html.
O*NET Career Exploration Tools
The Career Exploration Tools are based on a “whole person” concept and are designed
for career counseling, career planning, and career exploration. They include the Ability Profiler,
the Interest Profiler, and the Work Importance Locator. They also include electronic versions of
the Interest and Work Importance assessments—the Computerized Interest Profiler and the Work
Importance Profiler. These electronic versions can be downloaded onto a computer system and
used at no cost by schools, One-Stop Centers, and others. Most recently, with the release of the
My Next Move Web site, users can choose to complete a 60-item Web-based version of the
Interest Profiler. Users exploring careers may use the Ability Profiler to discover what they do
well, the Interest Profiler to identify the types of work they may like to perform, and the Work
Importance Locator to determine which occupations will likely be satisfying according to the
their values and needs. The tools enable users to discover important information about
themselves and use the information to explore the world of work. Workers may use these tools as
aids in exploring career options, in considering career transitions, and in preparing for career
change. The assessments also are designed for use by students exploring the school-to-work
transition.
O*NET Academy
Training support for the O*NET system can be found on the Web at
http://www.onetacademy.org. The O*NET Training Academy provides various user
communities (workforce development professionals, employers, education, students, WIBs, job
seekers, etc.) with access to O*NET support materials tailored to their needs. Through podcasts
and O*NET user spotlights, customers can gain new insights into user needs and real-world
applications of O*NET data. Through the O*NET Academy, O*NET users gain electronic
access to recorded Webinars, self-paced training courses, and best-practices tutorials on using the
O*NET system and applying it on the job. The Academy site also provides access to an array of
O*NET tools (e.g., O*NET OnLine, O*NET Questionnaires).
26
OMB Supporting Statement
Justification
O*NET Code Connector
The Code Connector (http://www.onetcodeconnector.org/) was developed to assist
workforce professionals needing to code jobs. One-Stop Career Centers, other government
workforce agencies, and college career services offices are the most prevalent users of the Code
Connector. The Code Connector uses the O*NET database to help users determine the correct
occupational code for their job orders. To access an occupation, the user may type in a keyword
or select an occupational group from the home page. On subsequent pages the user is able to
refine the search to select a specific occupation. The final report contains information to help
determine whether the selected occupational code is the best match for the particular job order.
This information includes the O*NET-SOC description, Tasks, Related Occupations, Occupation
Family, and Detailed Work Activities.
A.2.2 O*NET Web Site Statistics
Use of O*NET products has increased dramatically over the past few years. O*NET
OnLine currently averages over 1.1 million visits per month, up from the 775,000 visits per
month 3 years ago. The O*NET Resource Center (http://www.onetcenter.org) averages 305,000
visits per month—almost 4 times the number of visitors from 3 years ago. In addition, use of
career information systems, Web site linkages, user certifications, and O*NET product
downloads is widespread. By design, the primary dissemination strategy of the O*NET Program
is for the private sector to build O*NET-based products that are tailored to specific audiences or
user needs.
Career Information Systems
Private and governmental online career information systems using O*NET data and
career tools reach millions of customers annually. For example, Bridges
(http://www.bridges.com/us/home.html), a private provider of career information services, built
O*NET data into its system and is used each year by more than 1 million graduating high school
students in the United States and Canada. IntoCareers (http://www.intocareers.com/
products.htm), another career information delivery system (CIDS), serves more than 10,000
schools and state agencies in the United States. The content components of IntoCareers include
data on 508 O*NET-based occupations. State CIDSs also provide widespread access to O*NETdriven career exploration. Nationally, CIDS programs are accessed at more than 87,000 sites by
over 37 million users a year. The O*NET Program is designed to be accessible to multiple users,
in keeping with the directives from the Advisory Panel for the Dictionary of Occupational
27
Justification
OMB Supporting Statement
Titles.18 The O*NET Program encourages these and other service providers and product
developers to create applications that deliver O*NET information to the public.
Internet Linkages
According to an exploratory search conducted in April 2011,
more than 10,500 sites link to O*NET OnLine,
more than 700 sites link to the O*NET Code Connector, and
more than 2,500 sites link to the O*NET Resource Center.
There are many different types of organizations linked to the O*NET Web sites,
including the following:
libraries and career centers based in higher education;
higher education institutions’ schools of business, labor and industrial relations,
psychology, education, and counseling;
government agencies (primarily state labor-related agencies, some federal and local);
public libraries (especially those offering career and job search assistance programs);
career counselors, coaches, and recruiters (mostly private-sector vendors);
career exploration or job search assistance (both private and public sectors);
public school systems, educational associations, and secondary schools (often
recommending the O*NET database as a resource for faculty, parents, and high
school juniors and seniors);
human resources management;
law firms specializing in immigration law;
vocational rehabilitation, or occupational medicine and health; and
international sites in Turkey, Canada, the United Kingdom, Holland, Japan,
Bangladesh, and elsewhere.
User Certifications
When individuals or organizations download the O*NET database or intend to use all,
any one, or any part of one of the O*NET Career Exploration Tools to develop value-added
products, they are asked to voluntarily register their use by completing a certification form. As of
March 31, 2011, the certification database holds 3,346 entries. Because registration is voluntary,
this number likely reflects only a portion of those actually using O*NET products for their own
applications. An overview of those user groups that have submitted certification forms for their
use of O*NET products appears as Exhibit 7.
18
See Appendix C: Advisory Panel for the Dictionary of Occupational Titles: Final Report
28
OMB Supporting Statement
Justification
Exhibit 7. Main Organization Types Submitting O*NET Certifications
Organization Type
Percent
Government; public administration
30
Educational services
17
Computer systems design; programming services
7
Employment services
6
Temporary help services; employment placement agencies; human resource and executive search
consulting
5
Vocational rehabilitation services
4
Information and Internet publishing; broadcasting; software publishers
3
Health care
2
Individual and family services
2
Professional, scientific, and technical Services; legal services
1
Military
1
Other
22
Total
100
O*NET Product Downloads
From January 2002 to February 2011, downloads of O*NET products totaled 830,768
(Exhibit 8).The use of O*NET products and tools continues to increase. The O*NET Program,
through continuous improvement efforts based on user needs and advancing technology, works
to efficiently develop products that meet customer demands in both the public and private
sectors.
Exhibit 8. O*NET Product Downloads
Product
Database
Number of Downloads
82,968
Career Exploration Tools
Ability Profiler
173,744
Interest Profiler
201,085
Work Importance Locator
Computerized Interest Profiler, Work Importance Profiler Software
Total Career Exploration Tools
Other (e.g., Toolkit for Business)
Total O*NET Products
96,409
142,128
613,366
134,434
830,768
29
Justification
OMB Supporting Statement
A.2.3 Examples of O*NET Data and Products in Use
The O*NET Program provides comprehensive, up-to-date occupational information that
is used directly by the public through access to the O*NET Web sites, as well as indirectly
through value-added products delivered by both governmental and private developers. Provided
under the following general categories of product developers and service providers are examples
of how O*NET-based products benefit the public:
federal and state government agencies,
public workforce investment systems and workforce investment boards,
assessment and career information delivery systems,
educational and research institutions,
U.S. armed forces,
private companies and commercial products, and
international users.
The following examples are taken from the publication O*NET Products at Work
(http://www.onetcenter.org/paw.html). This publication is updated quarterly and provides a more
comprehensive listing of O*NET uses.
Federal and State Government Agencies
Federal and state agencies are using O*NET products to achieve their goals of service to
employers and the public. Business development specialists, human resources personnel, and
others rely on the common language of O*NET products to build connections required for a
strong workforce. Provided below are examples of O*NET products used by the Alabama
Department of Rehabilitation Services, the California Employment Development Department,
the Connecticut Department of Labor, the Maine Department of Labor, and the Montana State
Human Resource Division, among others.
The Alabama Department of Rehabilitation Services incorporated O*NET information
into their Employment Institute, a two-part training program required for everyone who provides
direct services to clients. Trainees are shown how to use O*NET OnLine to address the
vocational goals of clients, looking at tasks, skills required, work environments, and potential
barriers. Training also shows counselors how to use the O*NET Code Connector to link a
client’s past job title to an appropriate O*NET occupation
(http://www.rehab.state.al.us/Home/default.aspx?url=/Home/Welcome/Main).
To address the pressing need for skilled computer 3-D graphic artists and traditional
animators, California’s Employment Development Department conducted an industry study with
30
OMB Supporting Statement
Justification
the endorsement of the California Skillsnet Consortium. O*NET’s survey data collection
instruments and other materials were used in the study. Relying on O*NET data, human
resources personnel modified local training and education initiatives to help close the gap
between industry needs and local workers’ skills. Training programs defined the skills and
requirements for the evolving occupations of computer 3-D graphic artists and traditional
animators. State and local leaders forged important partnerships with local establishments,
educators, WIBs, community-based organizations, and other stakeholders to meet industry
demands and develop needed talent (http://www.doleta.gov/programs/ONET/ca-dc.cfm).
CareerOneStop, sponsored by ETA, offers job seekers, students, businesses, and
workforce professionals the most comprehensive set of career resources and workforce
information on the Web. O*NET products are integrated throughout the site, including the
O*NET database, O*NET Career Exploration Tools, and O*NET Tools and Technology (T2;
http://www.careeronestop.org/). In addition, mySkills myFuture, a new DOL-sponsored site
designed to assist dislocated workers by helping them to identify occupations that would make
use of transferable skills from their previous work, also relies on O*NET information as the basis
for the transferable skills matching algorithm.
ETA supports a Projections Managing Partnership (PMP) consortium that operates and
manages the nationwide program of state and local occupational employment projections. These
data are among the most requested and utilized datasets by a wide variety of workforce
information customers and private sector businesses. These datasets are used for multiple
purposes, including the preparation of WIA state plans for submission to DOL. The PMP
develops and makes available to all states a standard employment projections software suite to
facilitate the release of standardized estimates and output reports across states.
In addition to tools to produce both short-term and long-term employment projections,
this software suite also includes a skills-based projections (SBP) tool, particularly designed to
support workforce planning, policy, and decisions. The SBP tool integrates O*NET Knowledge,
Skill, and Generalized Work Activities data with a state’s or territory’s short- and/or long-term
occupational projections to identify current skill supply, projected demand, potential skill gaps,
and replacement needs.
The Connecticut Department of Labor used O*NET data to respond to the WIA
requirements that states assess current and future job opportunities in the state, the skills
necessary to obtain these jobs, and the skills necessary to meet the economic development needs
of the state. To meet these requirements, Connecticut collected and analyzed labor market
information and published an extensive report. O*NET Skills and skill descriptions were used in
the report sections describing skills necessary for Connecticut’s high-demand occupations and
31
Justification
OMB Supporting Statement
industry sectors important for state economic development
(http://www.doleta.gov/programs/ONET/ct.cfm).
The Maine Department of Labor uses O*NET information in their effort to attract new
business to the state. Businesses provide descriptions of the jobs they need to fill, and the Maine
Department of Labor searches O*NET data to match occupations to the knowledge, skills, and
abilities provided in the job descriptions. In addition, small businesses are helped to develop their
employees through training strategies based on O*NET occupational skills information.
O*NET information has also been used by the state of Maine to help dislocated workers
make informed decisions about employment opportunities after a layoff. On the basis of a
comparison of knowledge, skills, and abilities, the jobs the workers were leaving were first
linked to O*NET occupations to identify likely opportunities requiring similar skills. O*NET
information was then supplemented with state-level wage information, job openings, and
information about retraining possibilities so workers could choose re-employment opportunities
best suited to them.
O*NET data were also used by a boat builder in Maine who needed to move his
operation to reach a workforce with the new skills and technological knowledge required for a
changing industry. The Maine Department of Labor helped identify alternative locations by first
linking the skills required for boat building to O*NET occupations. Then, Labor Market
Information data were used to identify the location of workers employed in these occupations.
The company decided to relocate to one of those areas
(http://www.onetknowledgesite.com/spotlight_on_John_Dorrer.cfm).
The Montana State Human Resource Division relies on O*NET information for
employee classification using their broadband classification system. The detailed occupational
information in the O*NET database determines the most appropriate title and placement for a
given position. O*NET information is compared to the tasks in position descriptions and to
required competencies. State job titles and codes are aligned with the O*NET taxonomy to
expedite the use of O*NET data for classification and compensation. When government job titles
are difficult to match, O*NET task lists, knowledge, skills, abilities, and work activities assist in
making needed distinctions (http://hr.mt.gov/hrpp/classification.mcpx).
Public Workforce Investment Systems and Workforce Investment Boards
State workforce investment systems have always been among the primary users of
O*NET products. Serving employers and the public through state-sponsored online career
information systems and One-Stop centers, O*NET products are responding to the demand for
32
OMB Supporting Statement
Justification
information about high-growth industries and helping to build the connections needed for a
strong workforce.
The Dislocated Worker System, part of Alabama's Comprehensive Labor Market
Information System (ACLMIS), is a Web-based system providing assistance to individuals who
have lost their jobs because of permanent layoffs or plant closings. Its skill survey questionnaire
is based on O*NET data (abilities, knowledge, skills, and work activities). The system uses the
O*NET occupational classification system for easy linkage to Alabama’s labor market
information. The ACLMIS system allows users to research occupations, analyze their skills to
find suitable occupations, or find occupations similar to their current jobs
(http://www.doleta.gov/programs/ONET/alabama.cfm).
The California County of Humboldt Employment Training Division uses O*NET
occupational information to assist people with disabilities and those on public assistance.
O*NET Career Exploration Tools help clients get a comprehensive picture of their fit with the
world of work. Employers and potential employees are encouraged to consider the many facets
of work other than basic skills. The training division presents classes to employees of social
service branches, describing how to use O*NET OnLine to serve their clients
(http://co.humboldt.ca.us/HHS/SSB/EmploymentTraining.asp).
The Fairbanks Job Center in Fairbanks, Alaska, offers a weekly workshop, Planning
Your Career. The workshop draws young and old participants who explore careers by first taking
the O*NET Interest Profiler and Work Importance Locator. The O*NET Ability Profiler is
offered as well. The participants are given the O*NET Occupations Combined List to find
occupations that match their results. They are shown how to use O*NET OnLine to explore
occupations and select language for their résumés. Participants find that the O*NET Related
Occupations and Tools and Technology features are very helpful. One client had earned a
computer science degree in the 1970s, with a minor in geology. He looked at O*NET Tools and
Technology for geologists and discovered that the local university offered the courses and
training he needed to work in the occupation. Building the required knowledge sets and skills
while searching for a job in the appropriate O*NET occupations, he found a job as an entry-level
geologist (http://www.onetknowledgesite.com/spotlight_on_Tonja_Horn.cfm).
Texas CARES (Career Alternative Resource Evaluation System) is a Web site providing
career exploration and occupational information. Created by the Texas Workforce Commission,
the site offers versions of the O*NET Work Importance Locator and the O*NET Interest
Profiler, as well as Best Match, a skills transferability program that matches users to O*NET
occupations on the basis of their current skill sets. Occupational information for the many
33
Justification
OMB Supporting Statement
features available on the site is supported by the O*NET database
(http://www.texascaresonline.com/index.asp).
The Texas Workforce Commission has also employed the O*NET-SOC AutoCoder on
one of its Web sites. This Web tool provides high-quality occupational classifications for job
orders, unemployment insurance claims, and résumé searches. The AutoCoder also interprets job
titles written in Spanish and returns relevant O*NET-SOC codes. The O*NET-SOC AutoCoder
performs by splitting the text of a job opening, résumé, or unemployment insurance claim into its
individual words and phrases and makes matches to the words and phrases in the O*NET
database. Analysts have weighted the words in the database so that the most important words for
an occupation are given more importance in the match calculation. The O*NET-SOC AutoCoder
provides the means to make consistent and high-quality occupational code assignments
(http://autocoder.lmci.state.tx.us:8080/jc/onetmatch).
The Florida Agency for Workforce Innovation uses O*NET information for job
matching, re-employment analysis, and business recruitment. O*NET data are imbedded in
FRED, the Florida Research and Economic Database. One key application is preparing labor
supply studies for business recruitment. O*NET also plays a large role in Florida’s
Re-employment Labor Market Information One-Stop Workshop. The workshop trains workforce
development professionals to use the appropriate O*NET data sets and complementary tools for
re-employment analysis: selecting the right tools, evaluating current skills, discovering
transferable skills, studying related occupations, and finding growth industries. Workshop
participants learn how to overlay pertinent local labor market information on the data from these
dimensions of occupational information. The workshop enables staff members to help job
seekers, employers, and workforce professionals to meet their respective challenges
(http://fred.labormarketinfo.com/).
Indiana Workforce Development is responding to dramatic shifts in the economy by
helping workers transfer their skills from declining occupation to those that are in economic
demand. This agency uses the O*NET database as a primary tool for making multiple
comparisons that identify where knowledge, skills, and abilities overlap or where gaps exist
across occupations. When the northern Indiana recreational vehicle industry declined, the skills
of laid-off workers were compared with skills required to perform growth occupations in the
region. Then community colleges in the area could develop curricula to train displaced workers
in the skills that high-demand occupations require and that are currently lacking in the regional
workforce. Skills that workers transferred from their former jobs, paired with “gap skills”
developed through new training, prepared workers for jobs related to orthopedic products and
34
OMB Supporting Statement
Justification
medical devices manufacturing
(http://www.onetknowledgesite.com/spotlight_on_Hope_Clark.cfm).
Assessment and Career Information Delivery Systems
Organizations routinely rely on O*NET products as the basis for their assessment and
career information delivery systems. Community colleges, university career centers, and many
higher education Web sites also have integrated O*NET products into their career services to
students. From nonprofit organizations to high-powered consulting firms, specific examples
provided below indicate the range of organizations using O*NET products.
The American Foundation for the Blind (AFB) uses O*NET data to help the visually
impaired expand their employment possibilities. CareerConnect™ is a free employment planning
resource for persons who are blind or visually impaired. Sponsored by the AFB, the program
helps these individuals learn about the range and diversity of occupations available in the labor
market. It also provides mentors and information about assistive technology that can help them
do the work. This practical, user-friendly resource incorporates O*NET data to supply essential
information for career exploration and to expand the universe of jobs for persons with visual
impairments (http://www.afb.org/Section.asp?SectionID=7).
AZCIS is a career information system offered by the Arizona Department of Education.
The Web site provides education, career, and occupation information in both English and
Spanish to middle school, high school, college, and adult students. The O*NET career tools and
database are essential components of the career exploration process offered by AZCIS. Spanish
translations of the O*NET Interest Profiler and the O*NET Work Importance Profiler have been
available on this Web site since fall 2010 (http://www.azcis.intocareers.org/).
CareerScope, developed by the nonprofit Vocational Research Institute, enhances career
and educational planning processes, including career development portfolios, transition plans,
Individualized Educational Plans, and employee training and retention plans. Users include
secondary schools, community and technical colleges, state and community vocational
rehabilitation agencies, Veterans Affairs medical centers, and state adult and juvenile
correctional facilities. Written at a fourth-grade reading level, the valid and reliable interest and
aptitude assessment is self-administered and easy to complete. Accommodation options include
audio delivery, English with Spanish text, and untimed delivery. CareerScope’s comprehensive
reports incorporate O*NET occupational data and provide career recommendations based on the
client’s interest and aptitude results (http://www.vri.org/).
Cengage Learning has developed a product called Career Transitions that uses the
O*NET Interest Profiler and the O*NET database to help job seekers. Their Web site states that
career research is the number one use of public libraries today, sometimes stretching staff to their
35
Justification
OMB Supporting Statement
limits. Career Transitions helps public libraries manage the number of people seeking job search
help by providing a self-paced application that walks job seekers through the entire process from
assessing strengths and interests to exploring new job opportunities
(http://www.gale.cengage.com/careertransitions/).
A case manager for Michigan Works! has found My Next Move, with access to
information on more than 900 O*NET occupations, to be a very useful tool for her customers.
She works with the 55+ Program, a demonstration grant program that gives specialized
assistance to individuals who are 55 or older, unemployed, and seeking employment. Her clients,
the 55+ Program Navigators, are using the comprehensive, easy-to-use Web site as part of their
career exploration and job search activities (http://mynextmove.org/).
The National External Diploma Program (NEDP) is a program of the Comprehensive
Adult Student Assessment Systems (CASAS), which provides assessment and evaluation
systems for adult education and workforce programs in the public and private sectors. The NEDP
is a unique way for adult students to earn a high school diploma. Students who have acquired
many of their high-school-level abilities through work, family, and community experiences can
complete a high school diploma, opening the way to postsecondary education and improved
employment options. In July 2010, the NEDP selected the O*NET Career Exploration Tools as
the only official occupational tools for use by its students. Students using the career tools receive
results correlated to the up-to-date, detailed, and comprehensive occupational information
provided by the O*NET database (https://www.casas.org/home/?fuseaction=nedp.welcome).
Pathway Builder, a division of Profiles International Educational Services Division,
provides participants with an interactive, comprehensive, online experience to plan and monitor
their progress from high school through postsecondary education and into the workforce. The
Pathway Builder ePortal provides communication tools, educational modules, e-learning
programs, site-approved Web site connections, comprehensive reporting, and career exploration
tools. The assessments in the career planning ePortal solution provide users with a link to their
O*NET occupational matches. Additional features include job board technology and an e-mail
platform to assist with an education-training-workforce initiative to engage students and workers
with career success (http://www.pathwaybuilder.com/).
Women Employed, a national advocate for women’s economic advancement, developed
Career Coach, a free online career development tool that helps adults explore career options,
define career goals, identify education and training resources, and make step-by-step plans to
reach their goals. The Web-based program provides users with occupational data available from
the O*NET database. It also provides an online version of the O*NET Interest Profiler
(http://www.womenemployed.org/index.php?id=38).
36
OMB Supporting Statement
Justification
Higher education institutions are among the most prevalent types of users of O*NET
products for assessment and career information systems. A Web search shows O*NET products
at work in most state educational systems. A few examples include AlabamaMentor.org, Arizona
State University CRESMET, CaliforniaColleges.edu, Cascadia Community College
(Washington), University of Central Florida, CTMentor.org (Connecticut), IllinoisMentor,
Middle Tennessee State University, MississippiMentor, Normandale Community College
(Minnesota), PennsylvaniaMentor, TexasMentor, and Texas State University.
Educational and Research Institutions
With the help of the occupational information contained in the O*NET database, colleges
are developing educational curricula and research organizations are conducting a broad array of
research projects. Presented here are a few examples.
The AARP Public Policy Institute and the Urban Institute examined how job demands
have changed over time and what jobs might look like in the future. The authors of the paper
Employment at Older Ages and the Changing Nature of Work (R. W. Johnson, Mermin, &
Resseger, 2007, November) link detailed information on occupational characteristics from the
O*NET database to the March 1971 and March 2006 Current Population Surveys. They found a
sharp decline in the physical demands of jobs and the prevalence of difficult working conditions
over the past 35 years. They also found that jobs have become more stressful and cognitively
challenging. The implications of the study suggest that the decline in physical job demands
increases the opportunities for older people to remain at work. However, more stressful, more
cognitively demanding jobs complicate the picture. The study proposes that the changing nature
of work will likely promote old-age employment for most people or at least not interfere with
longer work lives.
ACT, formerly the American College Testing Program, conducted a study (ACT, 2006)
showing that high school students who plan to enter workforce training programs after they
graduate need math and reading skills similar to those of college-bound students. For the study,
ACT looked at O*NET job zone 3 occupations that
do not require a four-year college degree,
offer the potential for career advancement,
are projected to increase in the future, and
are likely to offer a wage sufficient for a family of four.
These occupations generally require some combination of vocational training and on-thejob experience or an associate’s degree. They include electricians, construction workers,
upholsterers, and plumbers. ACT then compared academic skill levels of profiled job zone 3
occupations with the College Readiness Benchmarks established for the ACT test. The results
37
Justification
OMB Supporting Statement
show that the levels of math and reading skills needed for success in the first year of college are
comparable to those needed by high school graduates to enter 90 percent of the profiled
occupations. On the basis of this study, ACT urges high school educators to offer students a
common academic program that prepares them for both postsecondary education and workforce
training programs (http://www.act.org/research/policymakers/reports/workready.html).
The Center for Hospitality Research of Cornell University conducted a study titled Ability
versus Personality: Factors that Predict Employee Job Performance (Tracey, Sturman, & Tews,
2007). The researchers used O*NET definitions, education and training requirements, tasks, and
required knowledge, skills, and abilities as part of their research to test the hypothesis that
cognitive ability is more important for predicting performance among inexperienced employees
than among experienced employees in frontline restaurant jobs. In addition, they hypothesized
that conscientiousness was more valid for predicting performance among experienced employees
than inexperienced employees. These hypotheses were supported in their research.
Central Michigan University (2004) has used O*NET Content Model domains to identify
the competencies for a leadership competency model for students. The model consists of five
dimensions, each with multiple competencies drawn from the O*NET Content Model and an
existing management taxonomy of work activities. Work requirements and worker
characteristics were reviewed to identify skills, knowledge, abilities, work styles, Generalized
Work Activities, work context, and organizational context relevant for each dimension. Although
the resulting model was intended for students and was distributed to members of the campus
community, it was also distributed to interested employers because of likely relevance to a
variety of organizations (http://www.chsbs.cmich.edu/leader_model/model.htm).
A study by the Federal Reserve Bank of New York, titled Human Capital and Economic
Activity in Urban America (Abel & Gabe, 2008, July), examined the relationship between human
capital (educational attainment and other knowledge measures) and economic activity in U.S.
metropolitan areas. The following is from their original study.
To arrive at the knowledge variables used in our analysis, we matched
occupational categories between the O*NET system and 2000 U.S. Census. In
many cases, we combined multiple O*NET occupations into a single Census
category. Following the general approach used by Ingram and Neumann (2006)
and Lakdawalla and Philipson (2007), we utilized the average value of the
knowledge importance or level across multiple occupations in the O*NET data.
With this information then available for 470 Census occupations, we calculated a
knowledge index that is the product of the knowledge importance and the
knowledge level. Feser (2003) used the same approach, noting that it places a
greater emphasis on high knowledge that is relevant to a given occupation
38
OMB Supporting Statement
Justification
(http://www.mnitworkforce.org/sites/mnitworkforce.advanceitmn.org/files/1.%20
NY%20Federal%20Reserve%20Report.pdf).
Harvard Business School students conducted a study (Hanna, 2008, December) to assess
the potential for offshoring more than 800 occupations represented in the U.S. economy. The
O*NET database served as the source of occupational information for the study on descriptors
such as tasks, knowledge, and skills. The student researchers considered the information in the
O*NET database and then rated occupations on a scale of 1 to 100 depending on whether the
occupations’ tasks could easily be performed offshore. The students estimated that between 21
and 42 percent of all U.S. jobs are potentially offshorable (http://hbswk.hbs.edu/item/6012.html).
A team of researchers under the auspices of the National Institute for Occupational Safety
and Health conducted a study to determine whether the O*NET database can be used to identify
measures for psychosocial and environmental factors in the workplace that can be linked to
national health surveys. Their goal was to examine associations between these workplace factors
and health risk behaviors and outcomes. Their study concluded that psychosocial, work
organization, and environmental job characteristics derived from the O*NET database can
successfully be linked to large-scale health surveys to examine these associations through a data
linkage method. The data linkage method used may provide other researchers with job
descriptors for use in research, surveillance, and workplace redesign. Linkages between health
data and occupational factors can provide surveillance data to monitor the changing associations
between working conditions and specific disorders for workers in different occupations. If
further research supports the methods and use of psychosocial factors derived from the O*NET
database, occupational health and organizational specialists may be able to use these linkages to
design and evaluate potential healthy workplace initiatives and interventions
(http://www.ncbi.nlm.nih.gov/pubmed/19092496).
CareerJournal.com, the Wall Street Journal Executive Career Site, conducted a study
designed to identify the “best careers,” those occupations for which workers report a high degree
of satisfaction. CareerJournal contracted the polling firm Harris Interactive to survey workers
about what jobs they had, whether they were satisfied, and what job qualities contributed to their
satisfaction. Four factors emerged for jobs held by highly satisfied workers: intellectual
stimulation, strong job security, high level of control and freedom of action, and extensive direct
contact with customers or clients. After identifying the common factors reported by highly
satisfied workers, CareerJournal used occupational data in the O*NET database, looking for
O*NET occupations with high ratings related to three of the four contributing factors. (Job
security was evaluated with BLS employment projections.) CareerJournal identified eight
occupations providing the most satisfying work experience:
39
Justification
curriculum and instructional coordinators,
high school special education teachers,
hospital and clinic managers,
management consultants and analysts,
medical researchers,
physical therapists,
sales, marketing, and advertising managers, and
OMB Supporting Statement
social workers, counselors, and related managers.
(“Best careers methodology—How we got our results,” 2006)
Martin Prosperity Institute conducted research to examine the relationship between wages
and the skills required in the workforce within a regional area. Using the O*NET database, they
performed a cluster analysis to identify three broad skill types—analytical, social intelligence,
and physical. They also conducted a regression analysis using O*NET data to quantify how each
skill contributes to regional prosperity. The institute found that analytical and social intelligence
skills have a significant positive relationship with regional wages, whereas physical skills have a
negative relationship. In addition, their research found that analytical skills are more closely
related to regional wages than are social intelligence skills. They also determined that analytical
and social intelligence skills increasingly result in positive impacts on wages over time and that
the positive impact of physical skills on wages has decreased over time
(http://research.martinprosperity.org/papers/Florida Mellander Stolarick Ross (2010) Cities
Skills and Wages.pdf).
Temple University’s Center for Professional Development in Career and Technical
Education in Pennsylvania supports professional growth and learning among all levels of
educators who work with career-bound students. The center offers certification and degree
programs for secondary career and technical education teachers, curriculum supervisors,
cooperative education coordinators, and career and technical school directors in the 17 counties
of eastern Pennsylvania. Center faculty and staff incorporate O*NET OnLine in their courses on
program planning and evaluation, curriculum development, and cooperative education. They also
use it as an aid in structuring occupational competency assessment committee reviews. With its
wealth of data on occupations, O*NET OnLine provides a reliable and readily accessible
resource for planning and evaluating programs and developing curricula in career and technical
education.
40
OMB Supporting Statement
Justification
The U.S. Armed Forces
The U.S. military has recognized the value of O*NET data and career tools in its various
transition programs, recruiting activities, and human systems development projects. Presented
here are a few examples of the range of O*NET products being put to work in the armed forces.
My Next Move for Veterans is designed for U.S. veterans who are current job seekers.
The interactive tool helps vets learn about their career options. The site has tasks, skills, salary
information, job listings, and more for over 900 different careers. Veterans can find careers
through keyword search; by browsing industries that employ different types of workers; or by
discovering civilian careers that are similar to their job in the military. Veterans can also take
advantage of the O*NET Interest Profiler, a tool that offers personalized career suggestions
based on a person's interests and level of work experience.
HireVetsFirst uses O*NET OnLine as a military skills translator to help veterans
returning from active military duty re-enter the workforce. Using their military occupational
classification in O*NET OnLine’s crosswalk function, they can identify equivalent civilian
occupations. They can view typical tasks, skills, knowledge, and abilities; look up wage levels
and employment outlooks, both nationally and in their own localities; and search for actual job
openings in areas of their choosing (http://hirevetsfirst.dol.gov/).
The Office of Wounded Warrior Care and Transition Policy (OWWCT), Transition
Assistance Program (TAP), is offering a variety of virtual learning opportunities to accommodate
the needs of the National Guard, Reserves, and wounded warrior populations and their families.
The virtual aspect of these learning modes opens up much-needed educational opportunities
often not available to service members because of their mobility, varying time zones,
accessibility, and stage of demobilization and integration. All that is required for attendance is a
broadband Internet connection and a telephone. Course registration is accessed via
www.TurboTAP.org. The series of 11 Webinar course offerings includes one on “Decoding
Military Skills for Civilian Employment,” which prominently features both O*NET OnLine and
My Next Move as tools for transitioning service members to use to facilitate a successful
transition to civilian employment.
The Military Spouse Resource Center (MilSpouse.org) is a Web portal created jointly by
the Departments of Defense and Labor for military spouses and other family members facing
career or educational needs. The site offers thousands of links to employment, training,
education, and other community resources. O*NET OnLine and the O*NET Interest Profiler are
key career exploration tools for spouses entering the job market. MilSpouse.org users can access
step-by-step guidance on using O*NET resources before beginning a job search, including
determining their skills and researching occupations of interest. Military personnel are also using
41
Justification
OMB Supporting Statement
the Web site when transitioning from military to civilian careers, using the Military Crosswalk
feature of O*NET OnLine (http://www.doleta.gov/programs/ONET/MilitarySpouse.cfm).
The Navy Manpower Analysis Center (NAVMAC) uses O*NET data in its work to
develop Occupational Standards (OCCSTDS). Navy OCCSTDS serve as a basis for training and
career development in areas such as formal school curricula, onboard training, development of
Personnel Advancement Requirements, and development of Navy-wide advancement
examinations. The O*NET skills taxonomy is used to categorize task statements as part of the
process to develop the OCCSTDS (http://www.npc.navy.mil/AboutUs/NAVMAC/).
The North Carolina Military Foundation teamed with the North Carolina Military
Business Center to create a database and interactive Web site that enables businesses to link their
needs to the competencies of troops exiting the military. One of the challenges faced by troops
and business leaders alike is identifying the knowledge, skills, and abilities shared by military
and civilian jobs. Using a keyword related to a job opening, employers are able to search for
related military occupations and information on how many military personnel in these
occupations are returning annually to civilian jobs. The user can view additional information
about these occupations, including a list of related civilian job titles. Further exploration is
available through a link to the related occupations in O*NET OnLine. This Web site helps
employers and transitioning military personnel come together through the common language of
the O*NET system (http://www.ncmbc.us/).
The Office of the Secretary of Defense enlisted the RAND National Defense Research
Institute (NDRI) to convene a panel of experts to provide assistance in refining the
implementation of the Department of Defense Human Capital Strategy (HCS). The goal of the
HCS is to develop a foundation for military personnel management. A major component of this
goal is a competency-based occupational analysis system. In the Final Report of the Panel on the
Department of Defense Human Capital Strategy (Hanser et al., 2008), the panel members
concluded that O*NET “has the potential to provide a framework for developing much of the
common language and functionality desired in a new DoD system.”
The U.S. Army Research Institute for the Behavioral and Social Sciences contracted with
the Human Resources Research Organization (HumRRO) to evaluate the O*NET system’s
usefulness for selection and classification purposes. The evaluation studied the following
O*NET Descriptors: Abilities, Skills, Generalized Work Activities, and Work Context. Army
subject matter experts as well as trained occupational analysts collected ratings on these
descriptors for several Army occupations.
Results of this study indicate that an Army occupational analysis system using O*NET
Descriptors as a foundation would provide a useful common language system with strong links
42
OMB Supporting Statement
Justification
to the civilian occupational database. Such links would provide substantial benefits for
recruitment and rapid mobilization efforts (Russell et al., 2008).
The U.S. Department of Defense’s Armed Services Vocational Aptitude Battery
(ASVAB) Career Exploration Program offers tools to help high school and postsecondary
students learn about career exploration and planning. Developers of the ASVAB Program
wanted to change its philosophy to emphasize wider career exploration and decision-making
among its participants. Completely redesigned, the program now uses O*NET data to broaden
occupational choices for nearly a million ASVAB participants annually at more than 14,000 of
America’s high schools. Students are encouraged to explore a variety of career possibilities
suited to their interests and learn to make decisions based on information about themselves and
about careers, instead of exploring just a few occupations that match their current abilities
(http://www.doleta.gov/programs/ONET/asvab.cfm).
Private Companies and Commercial Products
Private companies are using O*NET information for both in-house purposes (human
resources functions such as job description writing and employee development) and for
commercial product development. The number of products with “O*NET in-it” continues to
grow.
Adaptiqs, a software company specializing in survey and assessment development,
conducted research using the O*NET framework to test an adaptive approach to surveys and
assessments. The project used 5 O*NET questionnaires (Knowledge, Skills, Abilities,
Generalized Work Activities, and Work Context) to create a tool that, based on individuals’
responses to the questionnaires, provides a list of jobs that are most applicable to them. Because
these questionnaires together contain over 400 questions, the study used Bbn (neural networks)
to minimize, in real time, the number of responses required. On average, the Adaptiqs tool
required 53 questionnaire items to generate a list of the top 10 occupations related to the
respondent’s experience. The research also suggested that the Work Context questionnaire was
the most valuable and the Abilities questionnaire was the least valuable for classifying people
into the most probable occupations (http://www.adaptiqs.com/).
Assessment Associates International (formerly Human Resource Consultants) has
developed the Page Work Behavior Inventory (PWBI), an assessment to measure work styles in
employees. Work styles are important characteristics related to competencies that underlie work
performance. The PWBI assesses work styles as defined in the O*NET system and provides the
user with information that can be used to
focus and guide leadership development efforts,
pinpoint specific strengths that may help with career advancement,
43
Justification
OMB Supporting Statement
identify areas for training and skill enhancement,
identify work environments that may support success for the individual, and
help guide career choice and transition decisions (http://www.aaiassessment.com/products/page_work_behavior_inventory/).
CareerNoodle by Transcend Innovation Group provides an online suite of career
exploration and planning tools for middle school, high school, and college students.
CareerNoodle relies on modern learning theories and human-computer interaction principles so
that students are engaged, motivated, and in control of their career exploration process.
Interactive activities to measure student work interests and values are adapted from the O*NET
Career Exploration Tools. Occupational data come from the O*NET database. Transcend
Innovation Group’s David Burns says, “We use O*NET data and the career exploration tools.
...Thanks for providing us with such wonderful tools and information!”
(http://www.transcendinnovation.com/careernoodle.php)
Corporate Gray Online is a Web site devoted to linking employers with transitioning or
former military personnel. The online tool complements the organization’s military-to-civilian
career transition books and Corporate Gray job fairs. The Web site provides users with a link to
the O*NET OnLine Crosswalk Search. Users enter a military job code or title and may view
detailed occupational data on related civilian occupations (http://www.corporategray.com/).
Economic Modeling Specialists, Inc. (EMSI) has developed a system that allows users to
target competency gaps through a unique way of visualizing the makeup of occupations. Using
O*NET knowledge, skills, and abilities data, EMSI’s Strategic Advantage is a Web-based
analysis platform for use by workforce development and education professionals. It provides the
data needed to help these professionals make informed decisions necessary for regional growth
and development (http://www.economicmodeling.com/webtools/).
Choices® Planner by Bridges Transitions provides the tools needed for students to build
personalized plans for college and careers. An online career information delivery system,
Choices Planner integrates O*NET data, as well as the O*NET Career Exploration Tools, into
their online program. Bridges Transitions has included Spanish translations of all O*NET
components so the system can be used by both English and Spanish speakers
(http://www.bridges.com/us/prodnserv/choicesplanner_hs/index.html).
iNQUATE Corporation has incorporated O*NET data into its human-capital software
application. With a patent pending, iNQUATE is an integrated seven-module application that
accommodates the development and communication of corporate goals and objectives,
automates the performance management process, and engages managers in the best use of human
capital while these managers analyze costs associated with maintaining a workforce. iNQUATE
44
OMB Supporting Statement
Justification
integrates with onsite databases or information available from external providers through Web
services. All seven of iNQUATE’s modules work together to simplify the process and eliminate
duplication of information and effort (http://www.inquate.com/).
The Job Coach Re-employment Suite is an online suite of tools designed to minimize
unemployment duration and promote wage replacement. Developed by Hoover & Johnson, the
product’s focus is on getting the user back to work in an appropriate job as soon as possible. One
tool within the suite allows the user to inventory his or her total transferable skills in a skills
bank. Their skills bank is then compared with the skills required for O*NET occupations and the
resulting matches are linked to job listings in the user’s selected job search geographic area. The
Job Coach uses the O*NET taxonomy to cluster job listings from scores of job boards so the user
is able to identify employment openings related to the personalized skills bank. Special
approaches are present for subpopulations, including veterans, dislocated workers, long term
unemployment insurance claimants, ex-offenders, recipients of Temporary Assistance for Needy
Families and Aid to Families with Dependent Children, and recent job training graduates.
(http://hooverjohnson.com/login3.aspx).
Manpower, a worldwide provider of high-value staffing services with nearly 1,100
offices in North America and 4,500 offices in 80 countries, provides jobs to 5 million persons
every year and services more than 400,000 clients worldwide. The O*NET occupational and
skills taxonomy helps Manpower match the right person to the right job. The O*NET system
also offers a systematic structure that enhances Manpower’s analysis of the employment
marketplace and its tracking of staffing trends. By incorporating O*NET structure into its
procedures, Manpower has benefited by being able to
identify accurately the types of placements each field office makes;
locate field offices where the highest need exists;
more accurately consolidate information for various types of analysis, including
marketing analysis; and
begin the process of having Manpower offices in other countries map their
occupations to O*NET, enabling Manpower to more efficiently consolidate
information for global reporting
(http://www.doleta.gov/programs/ONET/Manpower.cfm).
Trustmark Insurance Company uses O*NET OnLine to collect information on job titles,
tasks, and skills and to access salary data. Human resources personnel use this information to
describe tasks associated with sales support activities and to align managers’ job titles with
appropriate occupations across industries. O*NET occupational descriptions and data help to
determine appropriate salary ranges for company positions and provide validation of company
salary survey data (http://www.trustmarkinsurance.com/internet/corporate/).
45
Justification
OMB Supporting Statement
Valpar International manufactures and develops specialized test instruments and software
that measure work-related skills. Their products are used worldwide in education, workforce
development, and allied health. Valpar’s Aviator software uses the O*NET database in
conjunction with various assessments to aid users in identifying occupations that fit their skills
and interests. The following description is found on their Web site.
Aviator’s multifunctional approach encompasses aptitude assessment, two pictorial/audio
interest surveys, and two databases - the standard and O*Net databases each with approximately
1000 jobs. Using Valpar’s skills-based assessment foundation, this fast, reliable, valid tool is
easy to administer and cost efficient. All subtests and surveys are computer-based and can be
completed in 60 minutes (http://www.valparint.com/index.htm).
International Users
O*NET data and career tools have quickly gained prominence in government and private
industry products around the globe. One example is Australia’s government, which uses the
O*NET database as a behind-the-scenes data set linked to their own Standard Classification of
Occupations. Human resource professionals in Japan have adopted the O*NET Career
Exploration Tools. Chinese researchers have relied on the O*NET database for their
occupational studies. European and Central American countries are translating O*NET products
for their own populations. O*NET OnLine has received visits from users in over 190 countries.
Countries logging hundreds of thousands of hits include Australia, Canada, China, Egypt, the
Netherlands, New Zealand, Puerto Rico, Singapore, Germany, Great Britain, Hong Kong, India,
Indonesia, South Africa, South Korea, and Taiwan. Each year, the O*NET Web sites log
millions of visitors from virtually every geographic region in the world.
CANEDCOM, a Canadian international education development company based in
Toronto, is working to install a Career and Vocational Guidance System for the Ministry of
Education in Oman in the Middle East. They are using the O*NET Interest Profiler as part of
their project to provide career guidance services for high school students, college and vocational
training center graduates, and other job seekers. After pilot tests resulted in dissatisfaction with
other assessments, CANEDCOM selected the O*NET Interest Profiler because it is an
“internationally accepted instrument that has been tested for reliability and validity”
(http://caneducom.com/index.shtml).
E-Career Guidance.Com, Ltd. (ECG) is an Irish company that provides vocational and
career counseling services. They developed an online career exploration tool, the
CareerDecisionMaker® (CDM), which assesses a client’s standing on variables from four
domains: vocational interests, work values, personality, and skills. The client’s score profile is
46
OMB Supporting Statement
Justification
matched to score profiles for occupations. The occupations having profiles that correspond most
closely to the client’s profile are presented as career options that the client might explore.
ECG linked the CDM to the O*NET database and its occupations from the Standard
Occupational Classification (SOC). Scores from two of the CDM domains link directly to
O*NET data for Interests and Work Values. The CDM skills and personality domains, however,
differ from the O*NET Skills and Work Styles domains (i.e., they are based on different
taxonomies). ECG contracted with Human Resources Research Organization (HumRRO) to
develop skills and personality profiles for O*NET’s SOC occupations that are commensurate
with the CDM skills and personality profiles. In addition, HumRRO devised a profile matching
algorithm that permits use of the entire four-domain CDM profile to identify promising career
options for ECG’s clients. All project activities (e.g., development of stimulus materials for
O*NET analysts, summary of personality ratings provided by O*NET analysts, data analysis,
development of the profile-matching algorithm) are documented in a technical report (McCloy,
Byrum, Munoz, & Tsacoumis, 2006). (http://www.careerdecisionmaker.com/ContactUs.aspx).
The Technical Education and Skills Development Authority (TESDA), a government
agency attached to the Philippine Department of Labor and Employment, has employed the
O*NET Ability Profiler in its aptitude assessments of hundreds of thousands of high school
students. In addition, TESDA is distributing the O*NET Computerized Interest Profiler to
regional, provincial, and district offices, as well as to their training institutions, as part of their eProfiling program, targeting out-of-school youth, career shifters, course shifters, and unemployed
adults (http://www.tesda.gov.ph/eTESDA/page.asp?rootID=3&sID=28&pID=12).
A.2.4 Examples of the O*NET Program in Published Literature
Presented here are some examples of references to the O*NET Program in publications.
For a thorough list of research articles, books, book chapters, technical reports, and presentations
referencing the O*NET Program, see Appendix D.
An article by Thompson and Koys (2010) in the Journal of Leadership and
Organizational Studies addresses improving the ability to set and assess learning outcomes for
students in an MBA or undergraduate management program. The article presents a protocol for
the development of a content- and skills-based management curriculum that is derived from the
Malcolm Baldridge National Performance Criteria and DOL’s O*NET database. Using the
protocol, the O*NET database defines specific needed skills and learning outcomes for students.
A researcher in the Department of Management and Organizations at the Tippie College
of Business in Iowa headed a team doing research to validate the WorkKeys Fit Assessment, a
tool which uses occupational interests and work values to measure person-occupation
congruence. The team used the O*NET database to develop and validate the assessment. Results
47
Justification
OMB Supporting Statement
of their work suggest that the fit between person and occupation predicts positive work attitudes
and outcomes. This relationship is a useful concept that can be applied to personnel staffing.
Organizations which include an interest-values assessment of person-occupation congruence
may increase job satisfaction, job match, and job commitment among their employees
(Postlethwaite et al., 2009, April).
At the Workshop on Research Evidence Related to Future Skill Demands, the National
Research Council Center for Education presented a paper, Projecting the Impact of Computers
on Work in 2030 (Elliott, 2007, May). This paper describes an approach to projecting new
workplace skill demands based on increased use of computers in the decades to come. An
analysis of the impact of computer technology on future skills used the O*NET database to
provide definitions, anchoring tasks for ability levels, and occupational ratings for the set of
human abilities that are broadly relevant to work. The results of the analysis suggested that a
serious and sustained effort is needed to project and prepare the American workforce for the
extensive changes that are likely to occur as computers continue to displace human activity in the
workplace.
The Research and Analysis arm of Indiana’s Department of Workforce Development
completed a study (Leeuw, 2006, December) designed to guide individuals, educators, and
workforce professionals in identifying occupations that share similar skills. The study used
O*NET skills data to create a new career cluster model that emphasizes relationships between
seemingly unrelated industries and occupations. These clusters are being used to prioritize and
focus resources on programs and curricula that support pathways to emerging high wage and
high demand occupations.
A book by Landrum (2009), Finding Jobs With a Psychology Bachelor’s Degree: Expert
Advice for Launching Your Career, describes how graduates with a degree in psychology can
develop a successful career with intellectual, emotional, and financial rewards. The book
includes profiles of professionals in the field, describing their work, degree of career satisfaction,
and how a bachelor’s degree in psychology is a factor in their work. Instructions are provided for
the use of interest inventories and career exploration tools such as the Holland Self-Directed
Search and the O*NET database.
A.2.5 Looking to the Future with Competency Models
DOL has identified industry and occupational competency models as a resource that is
key to businesses and industries articulating their workforce needs. The O*NET Content Model
and database provide a valuable resource for the development of reliable and valid competency
models. In turn, these models form the basis on which curriculum developers and training
providers develop objectives and outcomes to ensure that workers have the right skills. The ETA
48
OMB Supporting Statement
Justification
acts as a broker of information in the area of competency models and skill standards
development by promoting, supporting, and disseminating information
(http://www.careeronestop.org/competencymodel/). The information gathered through the
O*NET Data Collection Program continues to provide the workforce investment system partners
with tools necessary for investment in human-capital development.
To date, nineteen high-growth industry competency models have been developed, and
more are in progress. For some industries the information available from industry sources is
neither comprehensive nor readily available to developers. In these cases, model developers can
access occupational information in the O*NET database to identify the critical work
requirements, worker skills, and knowledge for occupations in the industry. The O*NET data are
an essential source of information, reducing the time and effort required to develop industry
competency models. When industry information is available or competency models have been
developed, a comparison with the O*NET data validates information from industry sources.
The University of Central Michigan has used O*NET Content Model domains to identify
the competencies for a leadership competency model for students. The model consists of five
dimensions, each with multiple competencies drawn from the O*NET Content Model and an
existing management taxonomy of work activities. Work requirements and worker
characteristics were reviewed to identify skills, knowledge, abilities, work styles, Generalized
Work Activities, work context, and organizational context relevant for each dimension. Although
the resulting model was intended for students and was distributed to members of the campus
community, it was also distributed to interested employers because of likely relevance to a
variety of organizations.
The Tennessee Valley Authority used O*NET information as presented in the Career
Pathways tool, which was designed by Economic Modeling Specialists, Incorporated, to evaluate
workforce competencies in northeastern Mississippi. By showing Toyota that competencies of
workers in the declining furniture industry in northeastern Mississippi were 98% compatible with
the needs of the auto industry, O*NET data helped bring a Toyota plant to northeastern
Mississippi.
A.3
Uses of Information Technology
The O*NET Data Collection Program employs the latest in information technology
systems and procedures to enhance the quality of the data, minimize the burden on the
responding establishments and job incumbents, and reduce the overall cost of the data collection
effort.
49
Justification
OMB Supporting Statement
A.3.1 Web Questionnaires
Electronic versions of the O*NET questionnaires are available via the Internet to sampled
job incumbents and occupation experts. Many of the benefits of the paper questionnaires are
replicated in the electronic questionnaires. Specifically, users are able to start and stop multiple
times without losing data. They can return to a partially completed questionnaire at any time
during the survey period and resume where they stopped. A respondent may also review and edit
previous answers as necessary. In addition, an on-screen progress meter keeps respondents
informed of their movement through the questionnaire.
Advances in Web technologies and security, as well as the increasing prevalence of
establishments’ and employees’ access to Web browsers, have made Internet-based data
collection both feasible and practical. Internet use continues to accelerate, and the use of the Web
questionnaire has increased with it. For 2004, 15.8% of employee respondents used the Web
questionnaire, a substantial increase over the 3.7% who responded via the Web during the 1999
pretest. In 2007 the number of job incumbent respondents choosing the Web version increased to
20.3%; for occupation experts, the Web usage rate was 24.0%.
The usage of the Web questionnaire continues to increase over time. For 2010, 24.8% of
job incumbents completed the survey online. For occupation experts, those using the Web-based
instrument in 2010 surged to 54.7%. The paper questionnaire cover and informational materials
mailed to respondents continue to highlight and encourage the option of completing the survey
on the Web.
The paper and Web versions of the questionnaires were designed to be optimal for their
respective modes of administration. The questionnaire design literature suggests that this
approach is essential to reduce mode effects. That is, if each questionnaire is designed to
minimize measurement error in its particular mode of interview, mode effects are also
minimized. For example, in the paper version, multiple questions appear on a single page of the
questionnaire. However, in the Web version, only one question appears per screen (although the
respondent can navigate at will through the instrument). The literature on Web survey design
(see, e.g., Couper, 2008) suggests that one question per screen for Web surveys reduces
measurement error and therefore the effects of administration mode. This change was the only
important one necessary for the Web version, because both instruments are self-administered. In
fact, to ensure comparability between the paper and Web responses, the formats and wordings of
the questions and response categories for the two versions remain identical.
A.3.2 Project Web Site
An O*NET data collection Web page application has been developed to support and
assist with the O*NET Data Collection Program (http://onet.rti.org). This site is divided into two
50
OMB Supporting Statement
Justification
major sections: the public and restricted-access sections. The restricted-access section is further
subdivided into two areas: the online questionnaires area and the project management area.
The goal for the public section is to support the establishment-recruiting process. This
section is accessible to the public, without restrictions. The public section includes O*NET
background information, endorsement letters, frequently asked questions, copies of the
questionnaires, and links to other O*NET Program–related Web sites. The purpose of this
section is to provide establishments, sampled workers, and occupation experts with readily
accessible information about the data collection effort and uses of the data.
The restricted-access section contains sensitive information available only to certain
groups, such as survey respondents and project managers. This section is controlled by a user ID
and password authentication scheme. The Web server includes a secure socket layer (SSL)
certificate to allow encrypted transmission of all information over the Internet. This same
technology is used by e-commerce Web sites to secure credit card numbers. In addition, no
“cookies” are used. (A cookie is data given to a user’s Web browser so that the browser will
return the data to the server or Web site during subsequent requests.) Some Internet users distrust
Web sites that deposit cookies on their computers and may even configure their computers to
prohibit cookies, so the “cookie-less” techniques ensure that the site will perform as expected,
whether or not a user has disabled cookies.
The online questionnaire area of the restricted-access section provides sample members
an alternative to pencil-and-paper self-interviewing. Only persons who have been selected to
participate in the survey have access to this area. Unique user IDs and passwords are assigned to
each job incumbent and occupation expert by a central office computer system at the time of
selection into the sample; names and other personally identifiable information are not obtained.
The ID and password are provided with other survey materials to the sample member. Before
allowing access to the online questionnaire area, the Web site confirms the validity of the ID and
password and verifies that a completed paper survey form has not already been received.
Having entered this portion of the site, respondents are
informed that participation in the survey is voluntary,
assured that their survey responses will remain private,
limited to seeing only the questionnaire they have been asked to complete,
permitted to stop at any point and continue responding later,
permitted to skip questions they choose not to answer, and
permitted to review and change previous responses.
51
Justification
OMB Supporting Statement
On the last page of the survey, respondents confirm that they have completed the
questionnaire and optionally enter comments about survey; then they exit the questionnaire area
of the site and are thanked for their participation. Their user ID is automatically deactivated at
this time. Any further attempts to log in will not be possible, because the system recognizes these
users as having completed the survey.
The database containing the survey data is not accessible via the Internet; it resides on a
server inside the contractor’s firewall. The Web data collection application encrypts and
transmits data from the respondent’s computer into the survey database. Only authorized project
staff, operating from inside the firewall, have access to the survey database.
The project management area of the restricted-access section contains data collection
management reports and information. Login credentials for this area are created for managers of
the O*NET Data Collection Program. This portion of the Web site serves as an intranet for the
O*NET Program, facilitating communication among contractor staff, program staff at the
National Center for O*NET Development, and DOL. Production reports are posted nightly and
include summaries of the progress of establishment recruiting, questionnaire shipment and
receipt, and overall data collection status. Additional applications include a secure, centralized
data file repository used by statisticians, analysts, and others to share results of specialized,
nonroutine analyses and reports. The project management area of the Web site reduces the cost
of the data collection effort by centralizing and streamlining features used by the project team.
A.3.3 The Case Management System and Data Collection Utilities
The O*NET CMS is a Web-based control system that supports and monitors the data
collection activities of the BLs, the mailing of informational materials and questionnaires, and
the receipt of completed paper and Web questionnaires. Enhancements have been made to the
CMS that allow greater flexibility and effectiveness in the communications between the BLs and
the POCs in sample establishments. For example, visual cues (icons) in the CMS allow the BLs
to prioritize and customize their approach for certain types of establishments that require special
procedures. Another enhancement is the ability to do ad hoc package modifications, such as the
inclusion of special endorsement letters, based on O*NET-SOC occupations. Because the
packages for specific O*NET-SOC occupations are customized, the POCs receive targeted
materials that help communicate the data collection mission. In addition, tools have been
developed to help operations management load-balance the number of cases assigned across the
team of BLs. Another feature, for use with multisite organizations, permits shipping of
questionnaires to more than one POC in an establishment.
An innovative RTI-developed approach to survey management known as Adaptive Total
Design (ATD) has been integrated into the CMS to provide real-time, graphical updates of
52
OMB Supporting Statement
Justification
process data captured during the execution of day-to-day operations.19 ATD ensures that
managers are alerted to and can swiftly take action on unusual situations that can arise. It also
allows managers to pair BLs to particular industries with which they have demonstrated
exceptional recruiting effectiveness.
In an effort to help manage supplies (envelopes, brochures, questionnaires, etc.)
associated with data collection, an inventory tracking system, integrated with the CMS, has been
developed and deployed. As the BLs place orders for informational materials and questionnaires
to be shipped to survey participants, the inventory system updates reports to show the expected
remaining inventory. As stocks run low, operations staff are alerted to replenish supplies. The
system provides a means to reconcile physical and expected inventory. As a result, the system
has improved the efficiency of ordering, storing, and shipping data collection materials.
Questionnaires are prepared for each respondent by an automated order-fulfillment
system. The system detects the questionnaire domain type and occupation assigned to the
respondent, and it prints a complete package, including the individually customized and labeled
questionnaire, ready to pack and ship.
The CMS automatically assigns questionnaire domain types sequentially as new
respondents are added to the sample. Depending on how many of each type are actually returned
by respondents, it is possible to receive enough responses of a particular questionnaire type for
an occupation before receiving the total desired number of questionnaires of all types. The
system continually monitors the number of returned questionnaires by type and overrides the
sequential assignment of questionnaires as appropriate so that only the questionnaire types that
are still required are sent. This feature enhances efficiency by eliminating the shipping of
questionnaires that are no longer necessary to complete the data for an occupation. Moreover, by
continually focusing resources only on the remaining questionnaire types required, it reduces the
total time required to complete any occupation.
A.3.4 Additional Uses of the Internet for Data Collection
The O*NET Program uses the Internet to gather additional occupational information,
such as lay titles and tools and technology. This use expands the O*NET database, providing
easily maintained current information about tools and technology and enhancing users’ ability to
find occupations relevant to their training and expertise by giving them a wider range of search
terms. This enhancement is accomplished without additional burden to the public and at less cost
than other means of data collection.
19
See Section B.4.10 for a description of ATD.
53
Justification
OMB Supporting Statement
“Web scraping” is employed to gather tools and technology linked to in-demand
occupations in high-growth industries, with particular emphasis on identifying rapidly changing
information technologies, such as database user interface and query software and analytical or
scientific software. Tools and technologies linked to specific occupations are found on job
posting, career education, professional association, and other Web sites. Occupational analysts
link the tools and technologies to the United Nations Standard Products and Services Code®, an
online classification system for tools and services. Before the information is published in O*NET
OnLine, rigorous reviews are performed to ensure the quality and usefulness of the data.
Information collected from the Web in this way represents a significant enhancement to the data
available to O*NET users.
O*NET lay titles are collected from unmatched search terms submitted to America’s
Career InfoNet (ACINet), a DOL-sponsored online resource for job seekers that helps
individuals explore career opportunities and make informed choices about employment and
education. O*NET lay titles, as well as tools and technology, are also collected from the ACINet
Job Description Writer, an online tool offered by ACINet to help employers and human resource
specialists write job descriptions. Search terms, such as Sports Medicine Specialist and
Nanotechnologist, that do not result in occupational matches in ACINet are captured
electronically. These terms, together with job titles collected through the Job Description Writer,
are reviewed by O*NET occupational analysts for potential inclusion in the O*NET Lay Title
database. Lay titles greatly enhance the keyword search functions for O*NET OnLine,
America’s Career InfoNet, and O*NET Code Connector, a Web application to assist workforce
professionals in matching job titles to O*NET-SOC codes.
Use of Web sites further enhances the O*NET database, provides greater search
capabilities to users, and reduces burden. It also allows for rapid update of the data through user
input and at minimal cost.
A.4
Efforts to Identify Duplication
To avoid duplication and reduce cost, several portions of the O*NET Content Model are
provided from existing data sources. Specifically, as discussed in Section A.1, the domain of
Workforce Characteristics—including information on industries, job opportunities, and pay—is
obtained through links to existing labor market information databases. Information about
occupational licensing, certifications, national industrial skill standards, and related instructional
programs is provided from existing sources, such as the Manufacturing Skills Standards Council,
the National Retail Federation, and several Web sites, including http://www.careertools.org/
certification and http://www.acinet.org/acinet/lois_start.asp.
54
OMB Supporting Statement
Justification
The exhaustive reviews of existing labor market and occupational information, conducted
by the Advisory Panel for the Dictionary of Occupational Titles and DOL’s DOT review staff, as
well as subsequent current research, identified no other comprehensive, valid, and reliable
sources that could be used for the data items included in the O*NET database. The development
of the O*NET Program has involved staff and advisors with many years of experience in labor
market and occupational information who are familiar with existing data sources. In fact, as
discussed in Section A.2, many existing systems that provide detailed occupational information
are actually using information based on O*NET data or the predecessor DOT.
The few existing sources with similar measures are too limited to be used in the O*NET
database. Some existing sources are valid and reliable—for example, information from the OPM
and the U.S. Department of Defense—but are not comprehensive, because they represent only
those jobs in federal civilian employment or the military. Some private sources of job analysis
information exist; however, they are based on job analyses conducted as needed rather than on a
representative sample of employers and workers. They are therefore limited in their coverage and
not representative of the entire workforce. Furthermore, these analyses are not comparable
because they do not use the prescribed O*NET common language to describe occupational
requirements; it is not practical to combine them, because they include dissimilar components.
Finally, these private data sources are not available to the general public.
A.5
Efforts to Minimize Burden on Small Establishments
All sizes of establishments are represented in the O*NET estimates for most occupations.
For some occupations, the targeting strategy used in selecting an efficient sample may lead us to
omit some small establishments from the sampling frame, but this omission occurs for few
occupations. The omission is allowed when it is clear that sampling small establishments will
greatly reduce the efficiency of the data collection or when it is clear that incumbents from small
establishments are not working in the mainstream of the occupation.
Given that establishments of all sizes should be represented in the samples for most
occupations, specific design provisions have been undertaken to avoid overly burdening small
establishments. For example, Exhibit 9 shows the distribution of establishments by number of
employees on the Dun & Bradstreet list used as the sampling frame for O*NET data. Exhibit 9
also shows the expected distribution of the O*NET sample of establishments by number of
employees. O*NET sampling selects small establishments at a much lower rate than that at
which they occur in the population: although 88.4% of establishments employ fewer than 25
employees, the O*NET sample will consist of only 38.1% of such small establishments. On the
other hand, large establishments (with 250 or more employees) will make up 24.3% of the
O*NET sample but make up only 0.3% of all establishments. Thus, the O*NET sample relies
55
Justification
OMB Supporting Statement
Exhibit 9. Distribution of Frame and Sample Establishments by Employment Size
Actual Distribution of
O*NET Sampled
b
Establishments (Percent)
Total Number of Frame
a
Establishments
Frame Distribution
(Percent)
1–4
10,815,608
69.0
14.1
5–9
1,823,001
11.6
6.0
10–24
1,227,005
7.8
18.0
25–49
458,259
2.9
8.4
50–99
254,976
1.6
15.0
100–249
135,776
0.9
9.4
250–499
31,102
0.2
13.3
500–999
11,557
0.1
6.2
6,476
0.0
4.5
425
0.0
0.3
907,022
5.8
4.9
15,671,207
100%
100%
Number of Employees
1,000–4,999
5,000+
Unknown
Total
a
b
Data based on January 2011 Dun & Bradstreet (D&B) frame of establishments.
Data based on distribution of prior O*NET samples that used the D&B frame. Future O*NET samples will be
similarly designed.
more heavily on large establishments in order to reduce the burden on small establishments with
few employees. The disproportionate sampling of large and small establishments is properly
accounted for in the analysis weighting, resulting in statistically consistent estimates.
In addition, data collection procedures place lower burden on small establishments than
on large establishments. When a small establishment is selected, it likely employs fewer of the
targeted occupations and has fewer employees working in the occupations. Thus, a POC at a
small establishment generally spends less time preparing sampling lists and distributing
questionnaires than a POC at a large establishment, which is more likely to employ several of the
targeted occupations, with a large number of employees working in the occupations.
A.6
Consequences of Collecting the Information Less Frequently
The O*NET database is the most comprehensive source of occupational information in
the United States. No other similar, comprehensive, reliable, and valid source exists. If O*NET
data are not collected, U.S. citizens, industry, business establishments, military, government and
educational institutions, and the workforce investment system will have few options that meet
their needs for occupational information. O*NET data are used to develop industry competency
models and occupational competency profiles. O*NET data also include information on
56
OMB Supporting Statement
Justification
transferable skills and are used for skills gap analysis, facilitating a mobile workforce responsive
to changing economic needs.
The use of O*NET data by industry, employers, software developers, job seekers,
students, educators, and workforce development specialists supports a well-functioning U.S.
labor market and workforce investment system—both essential to U.S. competitiveness in the
global 21st-century economy. O*NET-SOC occupations conform to the SOC, permitting
O*NET data to be linked to and analyzed with sources of information on current occupational
employment and trends, wages, and demographic data. Its electronic format is freely accessible,
either through O*NET OnLine (http://online.onetcenter.org) or as a download from the O*NET
Resource Center (http://www.onetcenter.org).
The initial 3.1 version of the O*NET database was populated with information derived
from the DOT. So far, updated data have been published for 1,133 O*NET-SOC occupations.
Additional data releases are planned through 2015 to update occupations and release new data on
new and emerging occupations. The consequences of discontinuing data collection would be that
the millions of users who rely on O*NET data for business and career decisions, for educational
programming, and for work in human resources or workforce development would instead be
using portions of information that are out of date and incomplete. If data were collected less
frequently, the currency of some data would become questionable, especially for occupations
that are changing as a result of new technologies. The focus of data collection on high-growth
and new and emerging occupations reflects the need to provide current information in a demanddriven economy that is rapidly changing.
A 3-year extension of the O*NET Data Collection Program is being requested for the
period spanning calendar years 2012 through 2015. This extension will provide for the updating
of selected high-growth occupations and for data collection activities for new and emerging
occupations. A dynamic and progressive U.S. economy requires continuous improvement to the
data on which so many decisions are based. Millions of people are currently using O*NET
information and the numbers continue to expand as public agencies and private developers
integrate O*NET data into their systems and products. The O*NET database provides valid,
reliable, and current occupational information crucial to a strong U.S. workforce. O*NET
database updates are scheduled to occur once a year to incorporate newly collected information
on recently surveyed occupations. A schedule for data analysis is provided in Section A.16;
schedules for data collection and analysis are subject to annual appropriations.
A.7
Special Circumstances
The study will be conducted in a manner consistent with the guidelines in 5 C.F.R.
§ 1320.9. There are no special circumstances that might require deviation from the guidelines.
57
Justification
A.8
OMB Supporting Statement
Consultation Outside the Agency
The Federal Register notice initiating the 60-day public comment period was published
on November 30, 2011. In response to this notice asking for public comment on the proposed
extension of the O*NET data collection, DOL received four comments that provided
observations about wage data for the SOC and O*NET code and title 11-9033, Educational
Administrators, Postsecondary. These comments made the observation that this category presents
problems for data users because it groups together different types of workers with different wage
levels into a single occupational estimate. The Department notes that the wage estimates for this
occupation are not produced through the O*NET data collection, but rather through an OES
survey conducted by BLS, which collects data for occupations based on the SOC categories.
Changes to the SOC classification are considered by the Standard Occupational Classification
Policy Committee (SOCPC), an interagency committee operating under the auspices of OMB
and chaired by BLS. These comments that were received in response to the Federal Register
notice for 1205-0421 will be forwarded to the staff supporting the SOCPC for consideration in
the next round of SOC revisions, tentatively scheduled to begin in late 2013. Interested parties
can find information about the schedule and process for any upcoming SOC revisions at
http://www.bls.gov/soc/#revision. Data users should note, however, that the 2010 SOC system
was issued by OMB on January 21, 2009, and is in use for data collection purposes. The most
recent solicitation for public comment on that version of the SOC was published in the Federal
Register on May 22, 2008.
The OMB clearance package was reviewed by two expert consultants, Drs. James B.
Rounds and Michael A. Campion. Both are industrial/organizational psychologists who are
widely published in their field and who are intimately familiar with the O*NET data collection
process and the types and uses of the data collected. Revisions responsive to their comments
have been incorporated, as appropriate, in the clearance package.
The data collection contractor, RTI International, has several mechanisms in place to
obtain ongoing feedback from study participants. The BLs have multiple telephone contacts with
POCs and occupation experts; they are careful to document in their call notes significant
comments or suggestions they receive. Subject matter experts at professional associations we
contact to help identify occupation experts frequently provide feedback that the National Center
for O*NET Development uses to refine occupation descriptions and tasks. Letters and brochures
sent to POCs and occupation experts offer a toll-free number they can call with questions or
comments. In addition, the project Web site, https://onet.rti.org/, which both study participants
and the general public can access, has a “contact us” tab that offers both a toll-free number and
an e-mail link. Furthermore, both the paper questionnaires and the online Web questionnaires
provide respondents an opportunity to submit comments with their questionnaire responses. All
58
OMB Supporting Statement
Justification
feedback received from these sources is promptly reviewed by project management staff and has
resulted in several modifications to the data collection protocol.
A.9
Payments or Gifts to Respondents
Since the origins of the survey, incentives have been offered to POCs, establishments,
and employees to encourage their participation in the O*NET Data Collection Program.
Although the current procedures are designed to minimize respondent burden, the effort for the
company and the POC participation is not insignificant.
A.9.1 Incentives for the Point of Contact and the Employer
The POC’s responsibilities include
reading the introductory package to become familiar with the purpose of the O*NET
Data Collection Program and the role of a POC;
seeking permission within the company, as necessary, to participate in the O*NET
Data Collection Program;
making a roster of all employees at the location who work in as many as five different
occupations;
participating in a sampling process that selects as many as 20 total employees from
these occupations and maintaining this sample roster for future reference;
distributing questionnaires to the sampled persons within the company and addressing
their questions and concerns about the survey; and
distributing follow-up materials to employees, including thank you–reminder cards,
replacement questionnaires, and e-mailed requests for cooperation.
Because POCs are the only link with the O*NET respondents, they must be fully
committed to the data collection process. The POC is the O*NET Program’s representative
within the establishment who communicates the importance of the O*NET Program.
The employer is also critical because he or she is being asked to
support the O*NET Data Collection Program by agreeing to the company’s
participation in data collection,
provide support and encouragement to the POC to carry out his or her responsibilities,
allow the POC to provide information regarding the number of persons employed in
the establishment in the occupations of interest, and
support the participation of the sampled employees.
Incentives for both POCs and employers are essential to encourage the highest response
rates possible. POCs receive a clock with an initial information package and, if they
subsequently agree to participate, a framed Certificate of Appreciation from DOL. The clock is
59
Justification
OMB Supporting Statement
battery operated, has the O*NET logo on its face, and is suitable for display on a desk or shelf.
The Certificate of Appreciation is printed in color on card stock, bears the DOL and O*NET
logos, displays the POC’s name, and is signed by a high-ranking DOL official. It has a solid oak
frame and Plexiglas cover and is suitable for displaying on an office wall. The total cost of both
gifts is approximately $10, although their perceived value likely far exceeds this amount.
The employer receives the O*NET Toolkit for Business if the establishment agrees to
participate. The Toolkit is an O*NET Program information packet, including a guide for writing
job descriptions, that managers can use for human resource planning. These materials are
attractively organized in colored, glossy folders.
The continuation of these incentives is planned for both the POC and the employer
because these incentives seem to work quite well and are the products of considerable thought
and experimentation. For example, in the early years of O*NET data collection, the POC
received a mouse pad with built-in calculator in place of the Certificate of Appreciation. The
mouse pad was not well received by the POCs, and its perceived value, with the clock, was
judged to be inadequate for all the POC’s efforts. Although not formally tested, the perceived
value of the Certificate of Appreciation has significantly exceeded its actual cost, and it has been
very well received by the POCs.
In addition, an experiment was conducted from 2002 to 2004 to evaluate an additional
incentive of $20 to the POC. Essentially, the incentive had no effect on POC cooperation rates or
employee response rates, but it significantly increased data collection costs. The experimental
findings suggested that the current POC and employer incentives are adequate for maximizing
response rates at a reduced cost (Biemer, Ellis, Pitt, & Robbins, 2006).20
A.9.2 Incentives for the Employee
In keeping with what has been done since 2001, each employee is offered a prepaid
incentive of $10 to ensure that a high percentage of the job incumbents respond by completing
the questionnaire.
Monetary incentives have the greatest potential impact when the respondent has to exert
some special effort, such as taking a test or filling out a multi-item questionnaire. The incentive
encourages respondents in a task requiring higher levels of involvement and commitment than
the typical one-time, face-to-face interview. Although the O*NET questionnaires are not tests,
the cognitive demands they place on respondents resemble test-taking demands in that the
respondents must assess the requirements of their job. The monetary incentive is instrumental in
impressing upon the respondent the importance of this rating task. Respondents who perceive the
20
For a more detailed discussion of the Biemer et al. (2006) results, see Section B.4.
60
OMB Supporting Statement
Justification
rating task as important will likely make thoughtful, carefully considered assessments rather than
hasty ones, thus improving the reliability of the data.
In addition, the monetary incentive is important because respondents are encouraged to
complete the questionnaire, which takes about 30 minutes, on their own time rather than on the
job. This encouragement not only minimizes the burden on employers, but also improves the
quality of the data because, otherwise, busy workers might be underrepresented in the data,
which would bias the estimates for job performance.
The monetary incentive may at least partially offset its inherent cost through efficiencies
created in the data collection process as a result of higher response rates. For the job incumbent
respondents especially—although they are not viewed as a difficult-to-reach population in the
usual sense—considerable effort and resources are expended to identify and reach them through
the sampling process. They represent a worker in a specific occupation in a specific
establishment in a specific industry. The expense of reaching that particular respondent justifies
the cost of a monetary payment to ensure a high rate of response. With regard to the size of the
employee incentive, payment amounts were evaluated in the pretest to determine the optimal
means to maximize the response rate. On the basis of these data, a $10 cash incentive has been
used since the initial wave of data collection and will continue to be used in future waves.
A.9.3 Incentives for Occupation Experts
Occupation experts provide data for approximately 25% of the O*NET-SOC
occupations. Each occupation expert receives a desk clock with the initial information package
and, if they agree to participate, a prepaid incentive of $40 and a framed Certificate of
Appreciation from DOL. Unlike job incumbents, who complete only one domain questionnaire,
occupation experts are asked to complete all three domain questionnaires; thus, the $40 incentive
is about $13.33 per questionnaire. This incentive is slightly higher than the $10 offered to
Establishment Method respondents for completing one domain questionnaire. The increased
incentive and the Certificate of Appreciation are necessary to gain cooperation from what is
often a rare group of experts for an occupation. Moreover, the additional incentives seem
commensurate with the effort involved in responding to multiple questionnaires, given that the
average occupation expert’s salary is higher than the average employee’s salary.
A.10 Assurance of Privacy
The contractor for the data collection task, RTI International, has extensive experience in
protecting and maintaining the privacy of respondent data collected from surveys. To ensure
privacy, RTI has drawn from its experience in designing the data collection procedures
incorporated in this program. In addition, RTI’s Institutional Review Board is bound by their
O*NET contract, federal regulations, and company policy to review and approve the research
61
Justification
OMB Supporting Statement
protocol to ensure compliance with federal regulations (45 C.F.R. § 46) concerning data privacy
and the protection of human subjects from research risks.
Survey data are collected from job incumbents (Establishment Method) and from
occupation experts (OE Method). Informational materials mailed to potential respondents
contain essential program information and assurances of privacy that enable the person to make
an informed decision about his or her voluntary participation in the data collection effort.
Examples of informational materials provided to survey participants appear in Appendix B.
Employees sampled at establishments are asked to complete their questionnaires on their
personal time, not company time. This stipulation enables the employee to select a comfortable
and private setting, if desired, in which to complete the questionnaire. All respondents have a
choice of completing paper questionnaires or completing the questionnaires online at the project
Web site. Paper questionnaires are mailed directly to RTI in a business-reply envelope provided
by RTI. The individual responses are processed according to a study ID number. All O*NET
Data Collection Program staff are required to sign a privacy pledge that assures each respondent
that the privacy of responses to the questionnaire will be maintained. Only authorized staff have
access to the completed instruments and data files. The completed and processed questionnaires
are stored in a secure document-control area until federal authorization has been granted to
destroy them. All computer files, including those associated with the control system, are
password protected.
The Internet-based system that allows respondents to provide their survey responses
electronically has restricted access, including a user ID and password authentication scheme for
respondents. The Web server includes an SSL certificate to allow encrypted transmission of all
information over the Internet, the same technology used by e-commerce sites. The database
containing the survey data is not accessible via the Internet; it resides on a server inside the RTI
firewall. Only O*NET Data Collection Program staff have access to the master survey database.
The O*NET questionnaires (see Appendix A) collect very little personal information
about the respondent, and what is collected contains no identifiers, such as personal name or
place of employment. No individual-level data are published, nor are they accessible or provided
to anyone except the O*NET Data Collection Program staff. Published results are made available
only in aggregate, as one set of estimates for an entire occupation. Furthermore, no demographic
data (e.g., sex, race) are released, even in aggregate form. Finally, estimates are not produced for
any subpopulations within an occupation, such as geographic region or sociodemographic group,
which otherwise may have allowed the identification of an individual.
Before publishing the O*NET tables on the public Web site, the O*NET Program team
thoroughly examines the tables for any risk of disclosure of private information. In particular,
62
OMB Supporting Statement
Justification
each table is analyzed to identify any “sensitive” cells (i.e., cells that may reveal too much
information about an individual employee). Willenborg and De Waal (1996) have recommended
using an (n,k)-dominance rule that a cell be regarded as sensitive if the sum of the largest n
contributions account for more than k% of the total cell value. Willenborg and De Waal further
recommend that n = 5 and k = 80. Because every sampled employee contributes only one
response, these recommendations translate into a minimum cell size of 5/0.80, or about 7. In fact,
the minimum cell size for the O*NET tables is 10 respondents; any sensitive cells with fewer
than 10 are suppressed. In addition, the (n,k)-dominance rule assumes a complete census. As
Willenborg and Waal note, when applied to tables based on samples and where the cell entries
are weighted averages, the (5,0.80) rule affords even greater disclosure control. This extra
control ensures that these O*NET tabular data pose no disclosure risks to any individual
respondent.
A.11 Questions of a Sensitive Nature
Only two questions in the O*NET questionnaires may be considered to be of a sensitive
nature. In the Background section, the survey uses the disability questions developed for the
2000 U.S. Census.21 The first of these questions asks respondents about sensory or physical
disabilities; the second asks about mental, self-care, go-outside-home, and employment
disabilities. Completion rates for these items indicate that the great majority of participants
(96%) elect to complete the questions.
The O*NET sampling strategy is to randomly select participants at the individual level.
The disability questions, together with the demographic questions, provide descriptive
information about the sample of respondents.
Respondents to the O*NET survey are informed that responding to all questions, which
includes disability status and the other demographic characteristics, is voluntary. They complete
the survey on their own time, in a private setting if they choose. No identifying information, such
as the respondent’s name or place of employment, is recorded on the questionnaire.
A.12 Estimates of Annualized Hour Burden
As described in Section A.1.3, there are two protocols for O*NET data collection—the
Establishment Method and the OE Method. An estimated 75% of occupations are completed by
Establishment Method. The OE Method is used for occupations as necessary to improve
sampling efficiency and avoid excessive use of burden, as when it is difficult to locate industries
21
The 10-item 2010 Census did not include disability items. Instead, these questions now appear in the American
Community Survey (ACS). The O*NET team is in the process of making the corresponding changes to its
questionnaires. Only a slight change in the wording will be needed.
63
Justification
OMB Supporting Statement
or establishments with occupation incumbents, employment is low, or employment data are not
available, as is the case for many new and emerging occupations.
The Establishment Method uses a two-stage sample, with establishments selected at the
first stage and employees selected at the second stage. Thus, there are burden hours associated
with both establishments and employees. For each participating establishment, a POC is
identified to coordinate data collection activities in the establishment. In Exhibit 10, the first and
second columns of the Establishment Activity section present the number of responses and
assumed average burden per response for the POC’s activities; these averages were obtained
from previous years of O*NET data collection experience. When the total establishment burden
was estimated, the estimated number of establishments that will complete each activity was
multiplied by the average burden and summed across the activities.
The Employee Activity section of Exhibit 10 is based on the fact that selected employees
under the Establishment Method will complete one domain questionnaire requiring an average of
30 minutes of effort, while occupation experts complete all three domain questionnaires,
requiring an average of 90 minutes of effort.
Exhibit 10 also displays the estimated number of sampling units and the estimated burden
hours. As shown in Exhibit 12 in Section A.16, we assume that 75 occupations will be completed
under the Establishment Method and 25 will be completed under the OE Method each year.
From June 2012 through May 2013, establishments are estimated to expend 6,247 burden hours,
and employees are estimated to expend 8,490 burden hours, for a total of 14,737 burden hours.
From June 2013 through May 2014, establishments are estimated to expend 6,113 burden hours,
and employees are estimated to expend 8,370 burden hours, for a total of 14,483 burden hours.
From June 2014 through May 2015, 5,172 burden hours are estimated for establishments, and
6,619 burden hours are estimated for employees, for a total of 11,791 burden hours. The slight
decline in total burden hours across the 3-year period of June 2012 to May 2015 (14,737; 14,483;
11,791) results from initiating data collection for most occupations during the period June 2012–
May 2013, which causes many of these occupations to complete data collection during the final 2
years.
The Total Respondents section of Exhibit 10 shows the estimated annual number of
respondents by category and overall. The respondent totals include the number of POCs (the row
name is “Screening call to POC”), the number of Establishment Method employee respondents,
and the number of OE Method respondents. The category totals are based on prior sampling
experience. The total number of respondents across all 3 years is 78,863.
The Total Burden Hours section of Exhibit 10 shows the annual number of burden hours
by category and overall. The burden hour totals include both establishment burden and employee
64
OMB Supporting Statement
Justification
burden. The category totals are based on the estimated number of respondents for each category,
the number of responses, and the average burden per response. The total number of burden hours
across all 3 years is 41,011. This 3-year total reflects a slight reduction in burden compared with
the previous 3-year period of 2009 to 2011, during which 43,857 burden hours were requested
(DOL, 2008).
O*NET Establishment Method data collection has been designed to minimize the burden
on the selected establishments:
Establishments are asked about no more than 10 occupations each, with questioning
terminated once 5, or sometimes fewer, occupations are identified at an
establishment.
Establishments are asked to complete rosters of employees only for the five or fewer
occupations identified.
Establishments are selected no more than once within a 12-month period.
No more than 20 employees are selected from an establishment across all of its
selected occupations.
If an occupation proves difficult to complete under the Establishment Method, the
dual-frame approach may be used to supplement it. For occupations that are difficult
to sample, the alternative OE Method may be used.
65
June 2012–May 2013
Number of
Average Burden
Responses per per Response
Sample Unit
(Minutes)
Sampling
Units
Burden Hours
June 2013–May 2014
Sampling
Units
Burden
Hours
June 2014–May 2015
Sampling
Units
Burden Hours
Justification
66
Exhibit 10. Estimate of Hour and Cost Burden by Year
Establishment Activity
Verification calls to initial
contact at establishment
1
2
15,500
506
12,833
Screening call to POC
Initial recruitment call to POC
1
3
1
12
12,588
629
10,652
533
7,301
1,460
6,178
1,236
POC creation of occupation
lists for Sampling
1
1,393
4,089
1,363
3,460
1,153
Call to POC to sample workers
POC’sdistributionof
questionnaire packets
4,179
696
4,089
681
3,460
577
3,928
982
3,843
961
3,252
813
Follow-up calls to POC
2
3,928
524
3,843
512
3,252
434
NA
NA
NA
6,247
NA
6,113
NA
5,172
Establishment Method
employee respondents
1
30
14,880
7,440
14,640
7,320
11,137
5,569
Occupation Expert Method
respondents
3
30
700
1,050
700
1,050
700
1,050
NA
NA
NA
8,490
N/A
8,370
NA
6,619
Private sector
NA
NA
11,611
NA
11,361
NA
9,613
NA
Federal government
NA
NA
934
NA
914
NA
773
NA
State/local/tribal governments
NA
NA
320
NA
313
NA
265
NA
Subtotal, establishment
respondents
NA
NA
12,865
NA
12,588
NA
10,652
NA
Individuals/households
NA
NA
15,580
NA
15,340
NA
11,837
NA
NA
NA
28,445
NA
27,929
NA
22,489
Total, establishment
a
517
15,167
12,865
643
7,462
1,492
20
4,179
1
10
1
15
4
428
Employee Activity
Total, employee
b
Total Respondents
NA
continued
OMB Supporting Statement
Total, all respondents
c,d
Number of
Responses
per Sample
Unit
Average
Burden per
Response
(Minutes)
June 2012–May 2013
Sampling
Units
Burden
Hours
June 2013–May 2014
Sampling
Units
Burden
Hours
June 2014–May 2015
Sampling
Units
Burden
Hours
Total Burden Hours
Private sector
NA
NA
NA
5,638
NA
5,517
NA
4,668
Federal government
NA
NA
NA
454
NA
444
NA
376
State/local/tribal governments
NA
NA
NA
156
NA
152
NA
129
Subtotal, establishment
burden hours
NA
NA
NA
6,247
NA
6,113
NA
5,172
Individuals/households
NA
NA
NA
8,490
NA
8,370
NA
6,619
NA
NA
NA
14,737
NA
14,483
NA
11,791
Total, all burden hours
e
OMB Supporting Statement
Exhibit 10. Estimate of Hour and Cost Burden by Year (continued)
Cost Burden
Respondent Type
Establishments
Employees
Total
g
f
6/12–5/13
6/13–5/14
6/14–5/15
$321,856
$320,986
$276,828
$245,446
$246,585
$198,760
$567,302
$567,571
$475,588
Note: NA = not applicable.
a
Includes total burden time for all establishments (private sector, federal government, and state/local/tribal governments).
b
Includes total burden time for individuals/households.
c
The total number of respondents across all 3 years = 78,863.
d
ThetotalnumberofrespondentswasderivedbysummingthenumberofPOCs(rownameis“ScreeningcalltoPOCs”),Establishmentemployees(rownameis
“EstablishmentMethodemployeerespondents”),andoccupationexperts(rownameis“OccupationExpertMethodrespondents”).
e
The total number of burden hours across all 3 years = 41,011.
f
Assumed hourly total compensation rates: $51.52 for June 2012–May 2013; $52.51 for June 2013-May 2014; $53.52 for June 2014–May 2015.
g
Assumed hourly total compensation rates: $28.91 for June 2012–May 2013; $29.46 for June 2013-May 2014; $30.03 for June 2014–May 2015.
Justification
67
Justification
OMB Supporting Statement
A.13 Annual Reporting Burden Cost
Exhibit 10 displays the estimated annualized cost to respondents for burden hours by
year. The cost burden was estimated with use of average total compensation rates obtained from
the December 2010 Employer Costs for Employee Compensation Summary issued by BLS on
March 9, 2011. The average total compensation per hour for private industry was $27.75, which
was inflated based on the Employment Cost Index to a median hourly total compensation of
$28.91 for June 2012–May 2013, $29.46 for June 2013–May 2014, and $30.03 for June 2014–
May 2015. These are the total compensation rates used for estimating the employee cost burden.
Given that the POC will often be a human resources manager, the December 2010 total
compensation rate of $49.46 for the Management, Professional and Related category was inflated
to a median hourly total compensation of $51.52 for June 2012–May 2013, $52.51 for June
2013–May 2014, and $53.52 for June 2014–May 2015 for estimating the establishment cost
burden.
Exhibit 10 shows that the combined establishment and employee total cost burden is
$567,302 for June 2012–May 2013, $567,571 for June 2013–May 2014, and $475,588 for June
2014–May 2015. As with the burden hours, the slight decline in total cost burden across years
results from initiation of data collection for most occupations during June 2012–May 2013,
which causes many of these occupations to complete data collection during the final 2 years.
There are no respondent costs for capital or start-up or for operations, maintenance, and
purchase of services. There are no costs to the employers, POCs, or sampled employees other
than the time it takes them to participate in the survey.
A.14 Estimates of Annualized Cost to Government
The estimated annual cost to the government for the O*NET Data Collection Program for
2012 to 2015 is approximately $4.6 million. This estimate includes all direct and indirect costs of
conducting the sampling, data collection, and analysis activities of the O*NET Data Collection
Program.
A.15 Reasons for Program Changes or Adjustments Reported in
Sections A.13 and A.14
Exhibit 11 compares the projected burden hours for the period June 2012–May 2015 with
the average annual burden as estimated for calendar years 2009–2011 in the OMB Supporting
Statement dated December 8, 2008. The projected total annual burden hours for June 2012 –May
2015 range from 11,791 to 14,737. The average annual burden is 13,670 hours, compared with
an average annual burden of 14,619 hours requested for the previous 3-year period (2009–2011).
In addition, as indicated in Exhibit 10, the total burden hours for the June 2012–May 2015
68
OMB Supporting Statement
Justification
period, 41,011, reflects a slight reduction in burden compared with the 2009–2011 period, for
which a total of 43,857 hours were requested (U.S. Department of Labor, 2008). Over the years
of conducting the O*NET Data Collection Program, experience gained in selecting efficient
samples of establishments has led to reduced burden hours. Continuous improvement efforts
include refinements in occupational clustering, industry identification, and the design of data
collection waves, as well as the ability to capitalize on previous sampling experience and history
for some of the occupations.
Exhibit 11 compares the estimated costs to respondents for June 2012–May 2015 with the
average annual cost estimated for 2009–2011. The annual costs have increased since 2009–2011,
primarily because of inflation in the benefits portion of employee compensation.
A.16 Time Schedule, Publication, and Analysis Plans
The major components of the O*NET Data Collection Program include sampling, data
collection operations, and analysis. Exhibit 12 shows the expected schedule for annual data
analysis cycles and data publications for the next several years.
A.16.1 Data Analysis Tasks Conducted for Each Cycle
Described here is the approach used for data cleaning and editing, as well as the analyses
that are performed annually.
Data Cleaning
Paper questionnaires are manually and machine edited so that completely blank
questionnaires are dropped; responses to items that should have been skipped are blanked out;
multiple responses are blanked out; and codes indicating missing data, multiple responses, and
legitimate skips are inserted. Codes for legitimate skips and missing responses are also inserted
in the records for questionnaires obtained through the Web. In addition, an electronic check is
conducted to detect duplicate questionnaires from the same respondent.
69
Annual Average 20092011a
Sampling
Burden
Units
Hours
June 2012-May 2013
Sampling
Burden
Units
Hours
June 2013-May 2014
Sampling
Burden
Units
Hours
June 2014-May 2015
Sampling
Burden
Units
Hours
Establishment Activity
Verification calls to initial
contact at establishment
14,861
495
15,500
517
15,167
506
12,833
428
Screening call to POC
12,335
617
12,865
643
12,588
629
10,652
533
Initial recruitment call to POC
7,154
1,431
7,462
1,492
7,301
1,460
6,178
1,236
POC creates occupation lists for
sampling
4,006
1,335
4,179
1,393
4,089
1,363
3,460
1,153
Call to POC to sample workers
4,006
668
4,179
696
4,089
681
3,460
577
POC distributes Questionnaire
Packets
3,766
941
3,928
982
3,843
961
3,252
813
Follow-up calls to POC
3,766
502
3,928
524
3,843
512
3,252
434
5,989
NA
6,247
NA
6,113
NA
5,172
15,759
7,879
14,880
7,440
14,640
7,320
11,137
5,569
Occupation Expert Method
respondents
500
750
700
1,050
700
1,050
700
1,050
Total, employee
NA
8,629
NA
8,490
NA
8,370
NA
6,619
Total establishment
Justification
70
Exhibit 11. Comparison of Hour and Cost Burden Between 2009–2011 and June 2012–May 2015
Employee Activity
Establishment Method
employee respondents
Total
14,619
Respondent Type
b
Establishments
Employees
Total
a
c
14,737
14,483
Annual Average
a
2009–2011
$281,483
June 2012–
May 2013
Cost Burden
June 2013–
May 2014
$321,856
$320,986
$276,828
$170,250
$245,446
$246,586
$198,760
$451,733
$567,302
$567,572
$475,588
From the OMB Supporting Statement dated December 8, 2008.
Assumed hourly total compensation rates: $51.52 for June 2012–May 2013; $52.51 for June 2013-May 2014; $53.52 for June 2014–May 2015.
c
Assumed hourly total compensation rates: $28.91 for June 2012–May 2013; $29.46 for June 2013-May 2014; $30.03 for June 2014–May 2015.
b
11,791
June 2014–
May 2015
OMB Supporting Statement
Total, Establishments and Employees
OMB Supporting Statement
Justification
Exhibit 12. Data Analysis and Publication Schedule
Analysis
Cycle
Analysis
Cycle Start
Date
Analysis
Cycle End
Date
Estimated
Establishment
Method O*NETSOC Occupations
Estimated
Occupation Expert
Method O*NETSOC Occupations
Estimated
O*NET-SOC
Occupations
Published
Publication
Date
14
Jul-12
Jun-13
75
25
100
Jun-13
15
Jul-13
Jun-14
75
25
100
Jun-14
16
Jul-14
Jun-15
75
25
100
Jun-15
Identification and Evaluation of Anomalous Cases
Analyses of the ratings are based on the assumption that raters were qualified, willing,
and able to engage in the rating tasks. This task is accomplished by requiring each case to first
pass through a series of machine edits using prescribed eligibility criteria, including having at
least one task rated important and having at least 50% of the domain questionnaire items
completed. Cases not meeting these criteria are excluded from the analysis file. Cases with
certain questionable characteristics are flagged in this editing process, and analysts review these
cases to determine their completion status. Flagged for review are all cases for which the
respondent (1) indicated in the “global match” item that the target O*NET-SOC occupation
description did not at all describe his or her own job, and (2) rated fewer than one third of the
tasks as important. An analyst reviews the self-reported job titles of these cases to determine if
they are at all likely to belong in the O*NET-SOC occupation. If a case does not belong, it is
removed from further analysis. If, in the judgment of the analyst, there is a reasonable chance
that the case belongs in the O*NET-SOC occupation, it is sent to the next stage of review.
Finally, cases that pass the machine edits and the analyst review are subjected to a
deviance analysis designed to identify cases that are outliers relative to other cases in their
occupation. The deviance analysis involves two procedures: (1) a statistical procedure to
quantitatively identify potential outliers and (2) an analyst review of these potential outliers to
make the final decision for each case. Cases that do not pass the analyst review are deemed
deviant within their occupation and are removed from the data set. On average, these activities
eliminate about 8% of all returned questionnaires. The cases passing all data cleaning criteria are
used to create the estimates for publication.
Computation of Sampling Statistics
Basic sampling weights are applied to the data to make inferences to the population of
incumbents for each occupation.22 These weights are computed as the inverses of the overall
selection frequencies and the selection probabilities for each selected establishment and each
22
For details about the weighting and estimation procedures, see Section B.1.1.
71
Justification
OMB Supporting Statement
individual participant. The analysis weights for the eligible sample units are adjusted to
compensate for unit nonresponse for both establishments and employees, multiple subwaves of
sampling, and sample adjustments. In addition, when the variation in the weights is large, the
weights are trimmed to reduce the variation. To maximize comparability of O*NET estimates to
estimates from other federal sources, the final sample weights are also ratio-adjusted to
occupation estimates obtained from the OES survey.
Sampling errors are computed. The analysis weights used in the sampling error
computations, as noted, have been adjusted for nonresponse and are consistent with the complex
sampling design.
Calculation of Descriptor Values and Reliability
For each occupation, the sample size, mean, standard deviation, and standard error of the
ratings for each descriptor are calculated, together with the 95% confidence interval around the
mean. Estimates with questionable precision are flagged and recommended for suppression if
any of the following conditions is true:23
The sample size (i.e., number of respondents who answered the question) is less than
10.
The variance is zero and the sample size is less than 15.
The relative standard error (RSE) is greater than 50%.24
In addition, estimates of sensitive data items with a cell count (i.e., the number of
respondents who selected a specific response option) of less than 10 are suppressed to ensure that
O*NET data pose no disclosure risk to any individual respondent.25 On average, approximately
2% of the estimates are flagged or suppressed.
Interrater Reliability and Agreement
Interrater reliability and agreement are assessed with three different analyses. The first
two measure reliability in terms of the covariation among ratings, while the third analysis reflects
rater agreement. The intraclass correlation coefficients (ICCs) for each item across all
occupations are computed. These results allow one to compare respondent rating variance within
an occupation with respondent rating variance across occupations. In addition to the ICCs, the
mean interrater correlations (Pearson’s r) are calculated for all pairs of raters within each
occupation. Finally, to assess the absolute difference among ratings of each item within each
23
O*NET suppression criteria are based on substantive expert recommendations (Peterson, Mumford, Borman, et
al., 1997) and on other large government surveys (see Klein, Proctor, Boudreault, & Tuczyn, 2002).
24
RSE = the standard error of the mean divided by the mean.
25
A sensitive data item is one that represents a value of a survey variable that an individual would not like to be
revealed and provides an unacceptable risk that an individual in the cell could be identified.
72
OMB Supporting Statement
Justification
occupation, an average deviation index is calculated for each item within each occupation. For
any given item and occupation, the average deviation index measures the average extent to
which each individual rating deviates from the item mean. Some differences in ratings within
occupations are expected because O*NET-SOC occupations comprise a range of different jobs in
most cases.
Each of the reliability analyses conducted (rater, standard errors) are influenced by the
number of respondents. The O*NET data collection methods include a sufficient number of
respondents in each occupation to ensure reliability (Peterson, Mumford, Levin, Green, &
Waksberg, 1997).
A.16.2 Creation of the Occupation Database
The O*NET database is scheduled to be updated annually. Each update will include data
for those occupations collected and analyzed during the previous 12-month period.
Consequently, a database update includes occupations from multiple data collection waves,
depending on the number of prior waves for which analysis was completed that year. For each
occupation collected, the newly calculated means data replace existing analyst-based or
incumbent-based data in the database. Users are provided with metadata regarding when the data
were collected and any other pertinent information that will assist the users in interpreting the
data.
The O*NET database is designated with a version number denoting each update (e.g.,
from O*NET 6.0 to O*NET 7.0). The database is developed and administered with the MySQL
database management system. Once the MySQL database is updated, it is used to generate the
database for public release as a series of flat text files. It is accessible to the public on the
O*NET Program Web site at http://www.onetcenter.org/.
A.17 Display of Expiration Date
The expiration date will be displayed on the cover of the survey questionnaires.
A.18 Exceptions to Certification Statement
There are no exceptions.
73
C. References
Abel, J. R., & Gabe, T. M. (2008, July). Human capital and economic activity in urban America.
Federal Reserve Bank of New York Staff Reports, 332.
ACT. (2006). Ready for college and ready for work: Same or different? Retrieved February 6,
2008, from http://www.act.org/research/policymakers/pdf/ReadinessBrief.pdf
Berzofsky, M. E., Welch, B., Williams, R. L., & Biemer, P. P. (2006). Using a model-assisted
sampling paradigm instead of a traditional sampling paradigm in a nationally
representative establishment survey. Proceedings of the American Statistical Association,
Section on Survey Research Methods (pp. 2763–2770). Washington, DC: American
Statistical Association. Retrieved from
https://www.amstat.org/Sections/Srms/Proceedings/y2006/Files/JSM2006-000811.pdf
Biemer, P., Ellis, C., Pitt, A., & Robbins, K. (2006). Effects on response rates and costs of a
monetary incentive for the point of contact in an establishment survey. Proceedings of the
American Association of Public Opinion Research. Phoenix, AZ.
Biemer, P., Ellis, C., Pitts, A., & Robbins, K. (2006). Effects on response rates and costs of a
monetary incentive for the point of contact in an establishment survey. Proceedings of the
American Association of Public Opinion Research.
Central Michigan University. (2004). Leadership competency model [O*NET database].
Retrieved January 2, 2008, from http://www.chsbs.cmich.edu/leader_model/model.htm
Chromy, J. R. (1979). Sequential sample selection methods. Proceedings of the American
Statistical Association, Section on Survey Research Methods (pp. 401–406). Washington,
DC: American Statistical Association.
Cicchetti, D. V., & Allison, T. (1971). A New Procedure for Assessing Reliability of Scoring
EEG Sleep Recordings. American Journal of EEG Technology, 11, 101–109.
Couper, M. (2008). Designing effective Web surveys. New York, NY: Cambridge University
Press.
Deville, J. C., & Särndal, C. E. (1992). Calibration estimation in survey sampling. Journal of the
American Statistical Association, 87(418), 376–382.
Dierdorff, E. C., Norton, J. J., Drewes, D. W., Kroustalis , C. M., Rivkin, D., & Lewis, P. (2009).
Greening of the world of work: Implications for O*NET-SOC and new and emerging
occupations. Raleigh, NC: National Center for O*NET Development. Retrieved from
http://www.onetcenter.org/dl_files/Green.pdf
Dillman, D. (1978). Mail and telephone surveys: The total design method. New York: Wiley.
Dillman, D. (2000). Mail and Internet surveys: The tailored design method. New York: Wiley.
Elliott, S. W. (2007, May). Projecting the impact of computers on work in 2030. Paper presented
at the Workshop on Research Evidence Related to Future Skill Demands Center for
Education, National Research Council, Washington, DC.
Fleishman, E. A., & Mumford, M. D. (1988). The ability requirements scales. In S. Gael (Ed.),
The job analysis handbook for business, industry, and government. New York: Wiley.
Fleishman, E. A., & Mumford, M. D. (1991). Evaluating classifications of job behavior: A
construct validation of the ability requirements scales. Personnel Psychology, 44(3), 523–
575. doi:10.1111/j.1744-6570.1991.tb02403.x
119
References
OMB Supporting Statement
Folsom, R. E., Potter, F. J., & Williams, S. R. (1987). Notes on a composite size measure for
self-weighting samples in multiple domains. Proceedings of the American Statistical
Association, Section on Survey Research Methods (pp. 792–796). Washington, DC:
American Statistical Association.
Folsom, R. E., & Singh, A. C. (2000). A generalized exponential model of sampling weight
calibration for extreme values, nonresponse and poststratification. Proceedings of the
American Statistical Association, Section on Survey Research Methods (pp. 598–603).
Washington, DC: American Statistical Association. Available from
https://www.amstat.org/Sections/Srms/Proceedings/
Folsom, R. E., & Witt, M. B. (1994). Testing a new attrition nonresponse adjustment method for
SIPP. Proceedings of the American Statistical Association, Social Statistics Section.
Washington, DC: American Statistical Association.
Hanna, J. (2008, December). How many U.S. jobs are "offshorable"? Harvard Business School
Working Knowledge newsletter., Retrieved December 8, 2008, from
http://hbswk.hbs.edu/item/6012.html
Hanser, L. M., Campbell, J., Pearlman, K., Petho, F., Plewes, T., & Spenner, K. (2008). Final
report of the panel on the Department of Defense human capital strategy. Report
prepared for the Office of the Secretary of Defense by the RAND Corporation.
Johnson, L., Jones, A., Butler, M., & Main, D. (1981). Assessing interrater agreement in job
analysis ratings. San Diego, CA: Naval Health Research Center.
Johnson, R. W., Mermin, B. T., & Resseger, M. (2007, November). Employment at older ages
and the changing nature of work (Urban Institute Report No. 2007-20). Washington, DC:
AARP.
Klein, R. J., Proctor, S. E., Boudreault, M. A., & Tuczyn, K. M. (2002). Healthy People 2010
criteria for data suppression. Hyattsville, MD: National Center for Health Statistics.
Korn, E. L., & Graubard, B. I. (1999). Analysis of health surveys. New York: Wiley.
Landrum, R. E. (2009). Finding jobs with a psychology bachelor’s degree: Expert advice for
launching your career. Washington, DC: American Psychological Association.
Leeuw, A. (2006, December). The butcher, the baker and the candlestick-maker revisited:
Indiana's new skills-based career clusters. INcontext, 7(12), Retrieved from
http://www.incontext.indiana.edu/2006/december/2006.asp
Mumford, M. D., Peterson, N. G., & Childs, R. A. (1997). Basic and cross-functional skills:
Evidence for the reliability and validity of the measures. In N. G. Peterson, M. D.
Mumford, W. C. Borman, P. R. Jeanneret, E. A. Fleishman & K. Y. Levin (Eds.), O*NET
final technical report. Salt Lake City, UT: Utah Department of Workforce Services
through a contract with American Institutes of Research.
Paxon, M. C., Dillman, D. A., & Tarnai, J. (1995). Improving response to business mail survey.
In B. G. Cox (Ed.), Business survey methods. New York: Wiley.
Penne, M., & Williams, R. (2007, July 30). Aggressive Weight Trimming Evaluation. Research
Triangle Park, NC: RTI International.
Peterson, N. G., Mumford, M. D., Borman, W. C., Jeanneret, P. R., & Fleishman, E. A. (1995).
Development of prototype Occupational Information Network (O*NET) content model.
Salt Lake City: Utah Department of Workforce Service through a contract with the
American Institutes of Research.
120
OMB Supporting Statement
References
Peterson, N. G., Mumford, M. D., Borman, W. C., Jeanneret, P. R., Fleishman, E. A., & Levin,
K. Y. (Eds.). (1997). O*NET final technical report. Salt Lake City: Utah Department of
Workforce Services through a contract with American Institutes for Research.
Peterson, N. G., Mumford, M. D., Borman, W. C., Jeanneret, P. R., Fleishman, E. A., Levin, K.
Y., Campion, M. A., Mayfield, M. S., Morgeson, F. P., Pearlman, K., Gowing, M. K.,
Lancaster, A. R., Silver, M. B., & Dye, D. M. (2001). Understanding work using the
Occupational Information Network (O*NET): Implications for practice and research.
Personnel Psychology, 54(2), 451–492. doi:10.1111/j.1744-6570.2001.tb00100.x
Peterson, N. G., Mumford, M. D., Levin, K. Y., Green, J., & Waksberg, J. (1997). Research
method: Development and field testing of the content model. In N. G. Peterson, M. D.
Mumford, W. C. Borman, P. R. Jeanneret & K. Y. Levin (Eds.), O*NET final technical
report. Salt Lake City: Utah Department of Workforce Services, through a contract with
the American Institutes for Research.
Peterson, N. G., Mumford, M. D., Levin, K. Y., Green, J., & Waksberg, J. (1999). Research
method: Development and field testing of the content model. In N. G. Peterson, M. D.
Mumford, W. C. Borman, P. R. Jeanneret & E. A. Fleishman (Eds.), An occupational
information system for the 21st century: The development of O*NET (pp. 31–47).
Washington, DC: American Psychological Association.
Peterson, N. G., Owens-Kurtz, C., Hoffman, R. G., Arabian, J. M., & Whetzel, D. C. (1990).
Army synthetic validation project. Alexandria, VA: U.S. Army Research Institute for the
Behavioral Sciences.
Petroni, R., Sigman, R., Willimack, D., Cohen, S., & Tucker, C. (2004). Response rates and
nonresponse in BLS and Census Bureau establishment surveys. Proceedings of the
Survey Research Methods Section (pp. 4159–4166). Alexandria, VA: American
Statistical Association.
Postlethwaite, B. E., Wang, X., Casillas, A., Swaney, K., McKinniss, T. L., Allen, J., & et al.
(2009, April). Person-occupation fit and integrity: Evidence for incremental validity.
Paper presented at the annual conference of the Society for Industrial and Organizational
Psychology, New Orleans, LA.
Roth, P. L., & BeVier, C. (1998). Response rates in HRM/OB survey research: Norms and
correlates, 1990–1994. Journal of Management, 24(1), 97–117.
doi:10.1177/014920639802400107
RTI International (Producer). (2004). SUDAAN language manual (Release 9.0).
Russell, T. L., Sinclair, A., Erdheim, J., Ingerick, M., Owens, K., Peterson, N., & et al. (2008).
Evaluating the O*NET occupational analysis system for army competency development
(Contract for Manpower, Personnel, Leader Development, and Training for the U.S.
Army Research Institute). Alexandria, VA: HumRRO.
Thompson, K. R., & Koys, D. J. (2010). The management curriculum and assessment journey:
Use of Baldridge criteria and the Occupational Network Database. Journal of Leadership
and Organizational Studies, 17(2), 156–166.
Tracey, J. B., Sturman, M. C., & Tews, M. J. (2007). Ability versus personality: Factors that
predict employee job performance. Cornell Hotel and Restaurant Administration
Quarterly, 48(3), 313-322.
121
References
OMB Supporting Statement
Treasury Board of Canada Secretariat. (2009, May). Public servants on the public service of
Canada: Summary of the results of the 2008 Public Service Employee Survey. Retrieved
from http://www.tbs-sct.gc.ca/pses-saff/2008/report-rapport-eng.asp#appa
RESOURCEID = 16339
Tsacoumis, S., & Van Iddekinge, C. H. (2006). A comparison of incumbent and analyst ratings
of O*NET skills. Alexandria, VA: Human Resources Research Organization.
Tulp, D. R., Jr., Hoy, E., Kusch, G., & Cole, S. (1991). Nonresponse under mandatory vs.
voluntary reporting in the 1989 survey of pollution abatement costs and experiments
(PACE). Proceedings of the American Statistical Association, Section on Survey
Research Methods (pp. 272–277). Washington, DC: American Statistical Association.
U.S. Bureau of Labor Statistics. (2011). Labor force statistics from the Current Population
Survey. Table 12: Employed persons by sex, occupation, class of worker, full- or parttime status, and race. Retrieved from http://www.bls.gov/cps/cpsaat12.pdf
U.S. Department of Labor, Employment & Training Administration. (2008). O*NET data
collection program: Office of Management and Budget clearance package supporting
statement and data collection instruments. Washington, DC: Author.
U.S. Office of Personnel Management. (2010). Federal Employee Viewpoint Survey.
Washington, DC: Author. Available from http://www.fedview.opm.gov/
Willenborg, L., & De Waal, T. (1996). Statistical disclosure control in practice series: Lecture
notes in statistics (Vol. 111). New York, NY: Springer.
Worden, G., & Hoy, E. (1992). Summary of nonresponse studies conducted by industry division,
1989–1991. U.S. Bureau of the Census. Washington, DC.
Workforce Investment Act, Pub. L. 105-220, 112 Stat. 936 (1998).
122
| File Type | application/pdf |
| Author | Loraine Monroe |
| File Modified | 2012-04-12 |
| File Created | 2012-04-10 |