O*NET Data Collection Program
OMB Control Number 1205-0421
OMB Expiration Date: November 30, 2024
SUPPORTING
STATEMENT FOR
O*NET DATA COLLECTION PROGRAM
OMB CONTROL NO. 1205-0421
This Information Collection Request (ICR) requests a revision to the Occupational Information Network (O*NET®) data collection program, to collect updated occupational characteristics and requirements information on an ongoing basis. Both selected sample sizes of business establishments and use of the Occupation Expert Methodology have been increased to offset declining response rates at both the establishment and employee levels; and the potentially lasting impacts of COVID-19 on business eligibility (see Section 15). Contacted Occupation Expert source organizations are presented with the opportunity of a recognition program for their participation (Section 9). Informational materials have been condensed to reduce redundancy and burden (Appendix E). Appendix F in this ICR package differs from the 2021 Appendix G submission: the look and format of the questionnaires have been modernized to reflect current best practices for surveys, including standardization of similar background questions between the Establishment and Occupation Expert questionnaires; and both the instructions for, and the presentation of level scale anchors have been updated to lend additional clarity. Also, the burden calculations incorporate up to 3 focus groups of size 30 each; these focus groups will be used to evaluate current protocols and/or materials and to guide future enhancements to O*NET data collection (see Appendix G and Supporting Statement Part B, Section 4.11.3). Finally, we propose to conduct incentive evaluation experiments at the establishment, employee, and occupation expert levels to assess the potential impact for reducing nonresponse bias, increasing response rates, and minimizing follow-up data collection efforts (Appendix I).
The O*NET Data Collection Program continually operates to populate and maintain a current database on the detailed characteristics of workers, occupations, and skills. The program uses an occupational taxonomy, the O*NET-Standard Occupational Classification (SOC), which has been updated to reflect the 2018 SOC, mandated by the Office of Management and Budget (OMB) for use by all federal agencies collecting occupational and labor market information. Previously, the O*NET-SOC was based on the 2010 SOC taxonomy. The O*NET-SOC comprises occupations at the most detailed level of the SOC and includes additional occupational specificity as needed. In addition, new and emerging occupations in high-growth sectors of the economy have been identified and added to the taxonomy. Data have been published for 853 O*NET-SOC occupations, 738 of which have undergone more than one update. The O*NET Data Collection Program received initial OMB clearance in 1999 for a pretest and eight subsequent clearances that have allowed data collection to continue without interruption since June 2001. The current clearance expires November 30, 2024. This request is to continue to update occupations that reflect older data as well as to collect data on new and changing occupations included in the 2018 SOC taxonomy for 3 more years, subject to annual budget levels.
O*NET information is the foundational common language for fulfilling the statutory and regulatory responsibilities regarding information on occupational skills discussed in the Workforce Innovation and Opportunity Act (WIOA) legislation and regulations.
Section 308 of the WIOA (Public Law 113-128) amended section 15 of the Wagner- Peyser Act to require the Secretary of Labor to oversee the “development, maintenance, and continuous improvement of a nationwide workforce and labor market information system,” which shall include, among other components, “skill trends by occupation and industry.” (See 29 U.S.C. 49l-2.) The O*NET program is the primary means for collecting skills information across all occupations in the economy. Updating the entire O*NET database is a critical component of the nationwide workforce and labor market information system to support employer, workforce, and education information needs.
The WIOA contains numerous references that identify the need for information on the skill requirements of jobs. The word “skill” is used more than 120 times, in addition to 3 references to “knowledge, skills, and abilities” and 2 to “knowledge, skills, and competencies.” For example, WIOA Section 102 requires the Unified State Plan to include an economic analysis of “(i) existing and emerging in-demand industry sectors and occupations and (ii) the employment needs of employers, including a description of the knowledge, skills, and abilities, needed in those industries and occupations.” (See 29 U.S.C. § 3112.) Section 134 requires the provision of “information on job skills necessary” and on “skill requirements” for obtaining jobs listed for the local occupations in demand. (See 28 U.S.C. 3174 § (c)(2)(A)(vi)(ii)).
Other WIOA references address the need for information on the skills of individuals. For example, Section 134 provides for assessment services to identify “the skill levels and service needs of adults and dislocated workers.” Section 129 allows funds to be used to provide youth with an assessment that “shall include a review of basic skills, occupational skills, prior work experience, employability, interests, [and] aptitudes.” (See 29 U.S.C. § 3164(c)(1)(A)). The O*NET Career Exploration Tools, including the O*NET Interest Profiler and O*NET Work Importance Locator, are such assessment tools, designed specifically to relate a person’s interests and work values to the information on education and skill requirements for occupations in the O*NET database and Web sites.
WIOA Section 303, on the Public Labor Exchange Services System, amends the Wagner- Peyser Act to add “The Secretary, in consultation with States, is authorized to assist the States in the development of national electronic tools that may be used to improve access to workforce information for individuals….” The suite of O*NET Web sites (such as O*NET OnLine and My Next Move) and O*NET Web Services used to disseminate O*NET occupational information and related workforce and labor market information are such national electronic tools designed to improve access to information for individuals.
Although the O*NET name is not referenced specifically in statutes, it is cited twice in the U.S. Code of Federal Regulations (C.F.R.). The WIOA regulations include definitions for O*NET and for O*NET-SOC, the taxonomy used for classifying O*NET data, as quoted below:
PART 651—GENERAL PROVISIONS GOVERNING THE WAGNER-PEYSER ACT EMPLOYMENT SERVICE
Sec. 651.10 Definitions of terms used in this part and parts 652, 653, 654, and 658 of this chapter. Authority: 29 U.S.C. 49a; 38 U.S.C. part III, 4101, 4211; Secs. 503, 3, 189, Pub. L
Occupational Information Network (O*NET) system means the online reference database which contains detailed descriptions of U.S. occupations, distinguishing characteristics, classification codes, and information on tasks, knowledge, skills, abilities, and work activities as well as information on interests, work styles, and work values.
O*NET–SOC means the occupational codes and titles used in the O*NET system, based on and grounded in the Standard Occupational Classification (SOC), which are the titles and codes by Federal statistical agencies to classify workers into occupational categories for the purpose of collecting, calculating, and disseminating data. The SOC system is issued by the Office of Management and Budget, and the Department of Labor is authorized to develop additional detailed O*NET occupations within existing SOC categories. The Department uses O*NET–SOC titles and codes for the purposes of collecting descriptive occupational information and for State reporting of data on training, credential attainment, and placement in employment by occupation.
Finally, the predecessor to the O*NET database, the Dictionary of Occupational Titles (DOT), is still cited in older regulations as a source of occupational information in support of federal programs. The DOT—which O*NET replaces— is cited 23 times in the C.F.R. (the DOT was formerly cited in statutes, but there are no current statutory references to it). These 23 citations include references to determining disability; administering Department of Labor (DOL) programs; and administering immigration, civil rights, and labor standards law. DOL officials responsible for the O*NET Program work with various federal users of the DOT, some of whom have made the transition either in regulatory changes or in practices and procedures, while others are exploring potential transition to O*NET information or uses of O*NET data. These include State Department officials responsible for visas, the Social Security Administration, and the Office of Apprenticeship, and the Office of Foreign Labor Certification within the U.S. Department of Labor.
Copies of the WIOA and 29 U.S.C. 49l-2, as well as the specific O*NET and DOT citations in the C.F.R., appear in Appendix A.
The continued population of the O*NET database is important because the O*NET database is the most current and comprehensive standard source of descriptive occupational information in the United States. The O*NET Data Collection Program remains at the center of an extensive network of occupational and skill information used by a wide variety of audiences, including individuals making career decisions; public agencies, such as workforce boards and American Job Centers, making training investment decisions; educational institutions preparing a future workforce; and employers making staffing and training decisions. With new workforce challenges due to the coronavirus (COVID-19) pandemic, O*NET data are an essential tool in assisting students, job seekers, employers, government agencies, and researchers in taking important steps to ensure a productive, successful economy.
This program provides a common language and framework to facilitate communication about industry skill needs among business, education, and the workforce investment system. The resulting O*NET database also is used to develop industry competency models and occupational competency profiles. O*NET data include information about transferable skills that are used for skills gap analysis, facilitating a mobile workforce responsive to changing economic needs. The O*NET database and companion O*NET Career Exploration Tools are used by many private companies and public organizations to tailor applications to their needs and those of their customers. The broad utility of the O*NET tools plays an important role in developing and maintaining a skilled workforce and contributes to U.S. competitiveness in a global, 21st-century economy. Additional information (e.g., content model and database history) regarding the O*NET Program can be found in Appendix B.1
The O*NET Program provides essential tools and services for numerous critical federal and state workforce investment functions. These tools also serve job seekers, businesses, and educational institutions across the country. The common language used in O*NET occupational and skill descriptions facilitates communication among the various user groups. This facilitation contributes to our nation’s talent development and promotes U.S. competitiveness in the global 21st-century economy.
Users of the O*NET database can access and use the information in various ways. The data are available to the public free of charge through the O*NET OnLine, My Next Move, and My Next Move for Veterans Web sites; through the O*NET Web Services application programming interface (API); or by downloading the database, which is done by developers who provide applications targeted to specific communities or audiences. The O*NET database is now the foundation for many programs serving the American workforce, providing information to build transferable skills, skills gap analyses, and competency profiles and to facilitate cross-functional team building. These examples are just a few of the ways the O*NET Program supports activities critical to maintaining a mobile workforce responsive to changing regional and national economic needs.
With the May 2023 version of the database, 853 O*NET-SOC 2019 occupations have been comprehensively updated with information collected from job incumbents and occupation experts through the O*NET Data Collection Program; 738 of these occupations have had more than one update. The O*NET database currently provides:
detailed occupational information for more than 900 occupations2;
information on standardized descriptors of skills, abilities, interests, knowledge, work values, education, training, work context, and work styles; and
occupational coding based on the 2018 SOC.
The O*NET database has been enhanced with occupational information that provides coverage of the occupations and will enable users to effectively use the database. These enhancements include (1) more comprehensive detailed work activity information; (2) updated military-to-civilian occupational crosswalks; (3) the identification of technologies employers most frequently include in job postings (e.g., hot technologies); and (4) the development of new alternate titles that enable users to more easily find occupations of interest in O*NET Web applications.
Easy-to-use competency frameworks for organizing workforce competencies, credentials, and other work-relevant information are available, including Work Activities, Technology Skills, Abilities, Cross-Functional Skills, Basic Skills, and Knowledge. The frameworks are published in Excel format and also as JSON-LD: machine-readable Linked Data described using the CTDL-ASN schema (Credential Transparency Description Language Profile of Achievement Standards Network Description Language) developed by the Credential Engine project.
A summary of O*NET machine-discoverable information can be found at the O*NET Resource Center’s Machine-Readable Data page.
The O*NET database is provided free of charge to the public through O*NET OnLine, a Web-based application at https://www.onetonline.org. O*NET OnLine offers users multiple ways to search for O*NET occupations and related data. O*NET OnLine includes:
regularly updated occupational information;
tiered search algorithms that maximize successful results of searches by keyword (title), occupational code, or partial code;
O*NET equivalents of occupations listed in other occupational systems, such as the Military Occupational Classification or the Classification of Instructional Programs;
the ability to browse by O*NET descriptor or variable (this search enables users to make cross-occupational comparisons by viewing an occupation’s rank order on a selected knowledge, skill, ability, work activity, interest, work value, tool or technology, or task);
the ability to browse groups of similar occupations, including Bright Outlook3; career clusters; industry type; or science, technology, engineering, and mathematics (STEM) disciplines;
a sample of reported job titles for each occupational report, providing the user with a broader understanding of the world of work;
report display options (in addition to the default Summary Report, users can choose to view a comprehensive Details Report or build a Custom Report).
Information below can be tailored to the local community that the user resides in or may be thinking of relocating to:
wage and employment trends information (derived from Bureau of Labor Statistics [BLS] data) and links to Web sites of professional associations, giving users access to additional information on related specialties, industries, and education and training resources; and
occupation-specific links to training, certification, licensing, and apprenticeship information, as well as to actual job openings.
My Next Move (https://www.mynextmove.org/) is an easy-to-use, easy-to-read Web-based interactive tool for new job seekers, students, and other career explorers to learn more about their career options. Users can explore more than 900 different careers and see important information about them, including skills, tasks, salaries, employment outlook, and related career videos. They can also look at related salary credentials and training, as well as search actual job openings. They can find careers by searching keywords, by browsing industries, or by using the O*NET Interest Profiler (see below). Additionally, users can browse occupations by Bright Outlook, interests, and amount of job preparation required.
Career reports in My Next Move feature the most important knowledge, skills, and abilities needed to perform the work, explained in language that is easy to understand. Outlook and education sections let users find salary information, job postings, and training opportunities. The visual design enables users to identify a career’s key points or to explore a career in depth.
Also, at the site is a Web-based version of the popular O*NET Interest Profiler, a tool designed to assess an individual’s vocational interests. The Web-based version of the Interest Profiler features 60 items that are scored and, along with information about the user’s education and work experience, guide users to careers they may enjoy. More information about the O*NET Interest Profiler–Short Form can be found at https://www.onetcenter.org/IP.html.
My Next Move for Veterans (https://www.mynextmove.org/vets/) is a Web-based interactive tool for U.S. veterans to learn more about their career options. This tool has all the functionality of My Next Move as described above.
Unique to My Next Move for Veterans is a feature that allows veterans to search civilian occupations related to their military training. Veterans indicate their branch of service and either the military code or title of their training. They then receive a list of civilian occupations that best match their military training. Where available, the results returned will also indicate how closely their military duties match the civilian occupations, along with the minimum military rank and length of military service associated with the civilian occupation.
The site’s Career Reports also offer rich information about military classifications through direct links into Careers in the Military (https://www.careersinthemilitary.com/). In addition, a new, interactive salary graphic is included within the Career Reports to aid veterans in understanding the often-wide range of salaries associated with civilian careers.
Mi Próximo Paso (https://www.miproximopaso.org/) is a Web-based interactive tool for Spanish-speaking job seekers, students, and other career explorers to learn more about their career options. Mi Próximo Paso includes all the features of the English-language site, My Next Move.
The Career Exploration Tools are based on a “whole person” concept and are designed for career counseling, career planning, and career exploration. They include the Interest Profiler and the Work Importance Locator. Within the My Next Move Web site, users can choose to complete a 60-item Web-based version of the Interest Profiler. A “mini” mobile-friendly version of the Interest Profiler was created and released for developers to leverage (https://www.onetcenter.org/reports/Mini-IP.html). A short, paper-and-pencil version of the O*NET Interest Profiler is now available for download and self-print, free of charge. The new tool is ideal for settings without computers or with very limited time for career assessment, such as correctional facilities and schools. The tool also has an updated Score Report and Career Worksheet, streamlined Career Listings, and a technical development report.
Individuals exploring careers may use the Interest Profiler to identify the types of work they may like to perform, and the Work Importance Locator to determine which occupations will likely be satisfying according to their values and needs. The tools enable users to discover important information about themselves and use the information to explore the world of work. Workers may use these tools as aids in exploring career options, in considering career transitions, and in preparing for career change. The assessments also are designed for use by students exploring a school-to-work transition.
The O*NET Resource Center (https://www.onetcenter.org/) serves as a central point of information about the O*NET program and is the main source for O*NET products, such as the O*NET database, Career Exploration Tools, and development reports. It also provides the latest news on O*NET developments and product releases, as well as important links to related O*NET sites.
The Code Connector (https://www.onetcodeconnector.org/) was developed to assist workforce professionals needing to code jobs. American Job Centers, other government workforce agencies, and college career services offices are the most prevalent users of the Code Connector. The Code Connector uses the O*NET data to help users determine the correct occupational code for their job orders. To access an occupation, the user may type in a keyword or select an occupational group from the home page. On subsequent pages the user is able to refine the search to select a specific occupation. The final report contains information to help determine whether the selected occupational code is the best match for the particular job order. This information includes the O*NET-SOC description, Tasks, Related Occupations, Occupation Family, and Detailed Work Activities.
O*NET Web Services was introduced to O*NET customers in January 2014. As of April 2023, there are more than 3,800 user accounts and approximately 20 million user requests per month. O*NET Web Services (https://services.onetcenter.org/) is an API that provides access to the full O*NET Database, along with the most popular features and tools from each of the O*NET Web sites. For developers, published APIs are available to connect vendor systems to key features of O*NET Web applications. Through O*NET Web Services, developers can integrate O*NET tools such as the following into their own Web sites or Web-enabled applications:
Keyword Search—both the My Next Move search and the OnLine occupation search are available for use in career sites. The REST Web Services API returns occupations matching a word, phrase, title, or full or partial O*NET-SOC code. The results include the code and title of each matching occupation.
My Next Move Career Reports—concise, easy-to-read overviews for over 900 occupations. APIs also provide Bright Outlook information, job outlook, and more.
Summary and Details Occupation Reports—detailed information from O*NET OnLine for more than 900 occupations. User applications can include an occupation’s most important or all tasks, knowledge, skills, abilities, technology skills, and more.
Military Search—the military transition search used in My Next Move for Veterans is also available through the Web Services API. The search returns relevant O*NET-SOC occupations based on full or partial codes and titles from the Army, Navy, Air Force, Marine Corps, and Coast Guard classification systems.
Spanish Keyword Search—the Spanish-language keyword search used in Mi Próximo Paso is part of the Web Services API. Occupation titles are returned, in Spanish, matching a Spanish word or phrase. A wide assortment of features from Mi Próximo Paso, including detailed career reports and Interest Profiler questions and scoring, is also available.
Interest Profiler—this assessment tool can be included in customer career tool sites using the IFrame Widget. After a simple block of HTML code is added, users can take the O*NET Interest Profiler without leaving their career resources Web site. For tighter integration, a REST Web Services API is offered. It provides scoring services and career results from the range of O*NET-SOC occupations. This tool is provided in both English and Spanish.
Browse by Career Cluster—career clusters contain occupations in the same field of work that require similar skills.
Browse STEM Occupations—STEM occupations require education in science, technology, engineering, or mathematics disciplines.
Browse by Industry—industries are broad groups of businesses or organizations with similar activities, products, or services. Occupations are considered related based on their employment within an industry.
Database Services—the API provides real-time, up-to-date access to all of the information in the downloadable O*NET Database. Users can retrieve data from any part of the Database and customize results by filtering and sorting based on multiple criteria.
Organizations using Web Services include federal, state, and local government agencies; military services; educational institutions; assessment and career information delivery systems; public workforce investment systems; private organizations and corporations; and international users. O*NET Web Services significantly reduce the cost and effort necessary for developers to update their applications with O*NET products and tools. With Web Services, O*NET data updates are seamlessly incorporated; thus, no new programing is required by developers. Additionally, as new features are added to O*NET Web applications, new Web Services are designed so developers can have immediate access to them and update their applications in an efficient and timely manner.
Use of O*NET products has remained strong over the past few years. O*NET OnLine currently averages more than 2.3 million visits per month. The O*NET Resource Center (https://www.onetcenter.org) averages more than 200,000 visits per month. The My Next Move sites average over 1.2 million visits per month. O*NET Web Services averages over 30 million user requests per month. In addition, use of career information systems, Web site linkages, user certifications, and O*NET product downloads is widespread; by design, the primary dissemination strategy of the O*NET Program is for the private sector to build O*NET-based products that are tailored to specific audiences or user needs. More information regarding Web site linkages, user certifications, and O*NET product downloads is available in Appendix Section C.1.
The O*NET Program provides comprehensive, up-to-date occupational information used directly by the public through access to the O*NET Web sites, as well as indirectly through value-added products delivered by both governmental and private developers. O*NET-based products benefit the public through product and service development by:
Federal and State government agencies,
public workforce investment systems and workforce boards,
educational and research institutions,
U.S. armed forces,
private companies and commercial product developers, and
international users.
As described in sections 2.1 and 2.2, the O*NET system provides ample opportunities to use O*NET products and tools. As previously mentioned, there are over 3,800 web service user accounts (https://www.onetcenter.org/ws_users.html) and several million monthly visits to O*NET web sites. Appendix C presents an overview of O*NET products and tools in use. Website and other user statistics are presented as well as examples of how O*NET has been incorporated and developed by a variety of public and private users.
The O*NET reference list (https://www.onetcenter.org/references.html) provides over 2,100 research and product-oriented references including research articles, books, book chapters and technical reports (previously Appendix D in the prior 2021 OMB packages). This list organizes the materials into six research areas: career vocational, education, health industrial organizational, labor market, other; and three product areas: application development, O*NET evaluation, and O*NET initiatives and development. The reference list, updated annually, can be browsed online or downloaded and printed. Presented here are two examples of references to the O*NET Program in publications.
To assess the health and economic impact of the COVID-19 pandemic on the Washington D.C. workforce, the D.C. Office of Planning used information from the American Community Survey and the O*NET 25.1 database (Funes, 2022). Using O*NET to group and contextualize occupations of D.C. residents, and the ACS to identify the number of workers in particular jobs, the researchers assessed the risk of job loss due to the spread of the pandemic. Results indicated that workers in professional and management sectors were least likely to lose jobs due to the pandemic. Workers in service, administration support, skilled labor and sales sectors had low to medium risk of job loss. Results from the research help D.C. government determine what re-skilling of the workforce was necessary to boost employment and what efforts were needed to mitigate loss of income by D.C. worker, including food and housing assistance.
The Rand Corporation, in an effort to help service members transition to the civilian world of work, developed a military to civilian occupation crosswalk using O*NET information as the foundation (Wenger et al., 2023). O*NET questionnaires were used to survey 5,100 service members about their military occupations. Results were linked to civilian occupation data included in O*NET Online (https://www.onetonline.org/). Focusing on entry level jobs, high quality civilian matches were ascertained using selected O*NET characteristics. Researchers found that there was significant overlap between their crosswalk and the matches provided in the O*NET program’s Military Transition Search (https://www.onetcenter.org/crosswalks.html#military). The authors also concluded that using O*NET questionnaires was beneficial in analyzing military occupations to help identify civilian matches.
The O*NET Data Collection Program is key to the continued effort to update the O*NET database to reflect changing skills requirements of occupations with the advent of new technologies and the changing world of work. In the research leading to the O*NET Data Collection Program, various sources and methods for collecting occupational information were examined, including collection of data from job incumbents and supervisors and development of ratings by occupation experts and occupational analysts. On the basis of this work, the O*NET team determined that the preferred source of data for most domains (Generalized Work Activities, Work Context, Knowledge, Education and Training, and Work Styles) is job incumbents. Other occupation experts, such as supervisors and trainers, may be used where access to job incumbents proves difficult or where the sampling of business establishments is inefficient.
By contrast, occupational analysts, who are provided with updated information from job incumbents, are preferred for the Abilities domain, which tends to be more abstract than the other domains. The Skills domain, whose variables are somewhat abstract, is a strong candidate for either source of collection; it is now updated by occupational analysts. A study conducted in 2006 found no clear evidence that one source of raters provides more valid or accurate data than the other for the Skills domain (Tsacoumis & Van Iddekinge, 2006).4 Consequently, considerations of relative practicality, such as cost, informed the decision to proceed with analyst ratings of both Abilities and Skills.
As part of a random sample, workers selected to participate in the O*NET Data Collection Program are asked to rate the requirements of their own jobs as defined by the O*NET questionnaire items. The responses are tabulated into statistics, such as mean ratings for each scale. Collecting information from job incumbents presents many challenges; among them is determining the best method for identifying a representative sample of job incumbents in each occupation. Sampling allows an estimate of the population. The O*NET Program is concerned, in particular, with identifying sampling approaches that minimize burden on employers and the public, achieving broad coverage of the workers in each occupation, ensuring acceptable response rates, and supporting overall cost-efficiency.
Three types of sampling frames are available for identifying samples of workers in each occupation: (1) lists of individual workers identified through professional and trade associations, licensing agencies, and unions; (2) households; and (3) employer establishments.
Identifying sampling frames of workers through professional and trade associations and unions retains the advantage of lower response burden because contacts with a sample of employers are replaced with contact with one or a few associations. Although it adds the cost of soliciting and maintaining association cooperation, this procedure also removes the cost of soliciting and maintaining employer cooperation.
The Establishment Method (using employer establishments to identify occupational samples, as described further under Sample Design below, as well as in Supporting Statement Part B, Section 1.1) provides the advantages of lower response burden and cost than a household survey, as well as good coverage for the majority of occupations. Response burden and costs are lower for two primary reasons: (1) there are more workers per employer than per household, so fewer contacts are required to identify workers; and (2) employer contacts can be minimized by focusing on those most likely to employ workers in each occupation for which the sample is required. Efficient sample design is possible because the distribution of employment in an occupation is usually a function of the industry of the employer. Employment by occupation by industry is measured by the federal-state Occupational Employment and Wage Statistics (OEWS) program national estimates provided by BLS.
The Establishment Method provides good coverage of wage and salary employment so long as an acceptable employer sampling frame is available. Coverage of self-employment is more difficult, although the sampling frame used in O*NET surveys includes many establishments operated by self-employed workers.
The Establishment Method, pretested in 1999 and 2000,5 continues to work well for many occupations. However, achieving high response rates with the Establishment Method can be challenging, because the method requires cooperation at two levels: the employer and the sampled worker. Though acceptable levels of cooperation have been attained to date, and this method has proved successful for many occupations, the O*NET team continually works to enhance response rates.6
An alternative method for collecting occupational information, involving occupation experts, is used to optimize the use of burden hours and resources because some occupations are difficult to sample and collect data from efficiently. This situation occurs for occupations with an extremely small population of job incumbents, new and emerging occupations or others lacking industry employment data, occupations whose incumbents were in remote locations that would have been difficult to access, or occupations for which use of the Establishment Method would be inefficient or ineffective. With the Occupation Expert (OE) Method, persons considered experts in the target occupation are surveyed. These experts include supervisors and trainers, as well as experienced job incumbents. The limitation of the OE Method is that locating experts can be difficult. For some occupations, identifying a professional association proves difficult; in other cases, the association may lack membership information sufficient to identify experts for a specific occupation.
Determining which sampling method should be used for an occupation is determined by comparing the advantages and disadvantages of the Establishment and OE Methods. As described below under Sample Design and in Section 15, use of the OE method is being increased to offset declining response rates at both the establishment and sampled worker levels.
Using the most appropriate sources of information (e.g., workers, occupation experts, analysts) and a multiple-method approach, the O*NET Data Collection Program efficiently collects and yields high-quality occupational data.
The O*NET Program sampling approaches are designed to create and update the O*NET database in a highly cost-efficient, timely manner while maximizing the reliability of the information in it. Historically, the primary method for collecting this information has been the Establishment Method, which entails a survey of workers employed in a national probability sample of establishments. To date, data collection has been completed using the Establishment Method for approximately 75% of occupations. The method uses a stratified two-stage design: (1) businesses (the primary stage) are selected from a national database, provided by Dun & Bradstreet (D&B), of roughly 18 million establishments, with probability proportional to the expected number of employed workers in the specific occupations being surveyed, and (2) a sample of workers (the secondary stage) is selected in the occupations within the sampled businesses. For selected occupations that are difficult to complete or for which additional observations are required, a special frame, such as a professional or trade association membership list or an alternate frame of establishments, is sometimes used in lieu of or to supplement the D&B sample. The sample selection procedures vary across special frames, depending on the type of information available on frame members. In general, association lists are sampled with a single-stage, stratified, simple random sampling approach. Stratification by geographic location and occupation subspecialty is considered if it is appropriate for the occupation.7
The OE Method is considered for use when the Establishment Method would likely be problematic because the target occupations have very low employment rates, are new or emerging occupations lacking industry employment data, are populated by incumbents in remote or difficult-to-access locations, or when it is difficult to locate or solicit participation of establishments with occupation incumbents. The OE Method is generally only used if the occupation is well represented by one or more professional or trade associations that are willing and able to identify experts in the target occupation. Occasionally, if related professional associations decline to participate, alternate sources of experts (e.g., commercially available lists of licensed professionals in the occupation) may be used. For this method, stratified samples of experts are selected from lists of potential respondents. These potential experts are questioned to determine whether they meet the program-specified criteria to serve as occupation experts for their respective occupations. Historically, data collection for approximately 25% of the O*NET-SOC occupations has been completed by the OE Method.8 With the need to increase the use of the OE Method to help offset declining response rates with the Establishment Method, it is estimated that this could increase to 40% by February 2027.
The O*NET Data Collection Program employs the latest information technology systems and procedures to enhance the quality of the data, minimize burden on the responding establishments and questionnaire recipients, and reduce the overall cost of the data collection effort.
Data collection operations are conducted by RTI International at its O*NET Operations Center in Raleigh, North Carolina, and its Research Operations Center, also in Raleigh. For the Establishment Method, the Operations Center’s Business Liaisons (BLs) contact sampled business establishments, secure the participation of points of contact (POCs), and work with the POCs to carry out data collection in the target occupations. The data are provided by randomly selected employees in the occupations of interest. All within-establishment data collection is coordinated by the POCs; the BLs do not contact employees directly. After a POC agrees to participate, informational materials and questionnaires are provided to the POC, who distributes the questionnaires to the sampled employees. Completed questionnaires are returned directly to RTI for processing. Respondents also have the option of completing the survey online.
As noted above, for difficult-to-complete occupations, the D&B sample may be supplemented with a sample of workers selected from a professional or trade association membership list, or a sample of establishments from an alternate frame (e.g., a list of nuclear power plants in the U.S.). Similarly, when the OE Method is used, occupation experts are also selected from professional or trade association lists. In both situations, the workers or occupation experts are contacted directly by the BLs, without involvement of a sampled establishment or a POC.
Three domain questionnaires are used to collect data from sampled workers: Knowledge (including Work Styles and Education and Training), Generalized Work Activities, and Work Context. Each sampled worker is randomly assigned one of the three questionnaires. The workers are also asked to provide basic demographic information and to complete a brief task inventory for their specific occupations. By contrast, the occupation experts are asked to complete all three domain questionnaires, as well as basic demographic questions and a task inventory for the occupation of interest.
Workers may complete the paper questionnaire and return it by mail, or they may choose to complete the questionnaire online at the project Web site. Occupation experts have the same options for completing their questionnaires. Questionnaires are available in Spanish for selected O*NET-SOC occupations. Data for two domains, Abilities and Skills, are provided by trained analysts because of the more abstract nature of the questions.9
Electronic versions of the O*NET questionnaires are available via the Internet to sampled job incumbents and occupation experts. Many of the benefits of the paper questionnaires are replicated in the electronic questionnaires. Specifically, users are able to start and stop multiple times without losing data. They can return to a partially completed questionnaire at any time during the survey period and resume where they stopped. A respondent may also review and edit previous answers as necessary. In addition, an on-screen progress meter keeps respondents informed of their movement through the questionnaire.
Advances in Web technologies and security, as well as the increasing prevalence of establishments and employees’ access to Web browsers, have made Internet-based data collection both feasible and practical. Internet use continues to accelerate, and the use of the O*NET Web questionnaire has increased with it. For 2019, 42.0% of job incumbents and 71.9% of occupation experts used the online survey. For occupations with historically high levels of online web response, efforts are undertaken to reduce printing, shipping and data entry costs by minimizing the need to send a paper questionnaire initially. In these instances, a paper questionnaire can be requested through the POC, or a paper questionnaire will be sent as part of subsequent nonresponse protocol procedures. The paper questionnaire cover and informational materials mailed to respondents continue to highlight and encourage the online option.
The paper and Web versions of the questionnaires have been redesigned to be optimal for their respective modes of administration and modernized to incorporate current best practices. The questionnaire design literature suggests that this approach is essential to reduce mode effects. That is, if each questionnaire is designed to minimize measurement error in its particular mode of interview, differences between the two types of questionnaires (Web and paper) are also minimized. For example, in the paper version, multiple questions appear on a single page of the questionnaire. However, in the Web version, the domain questionnaires typically display only one question per screen (although the respondent can navigate at will through the instrument). The literature on Web survey design (see, e.g., Couper, 2008) suggests that one question per screen for Web surveys reduces measurement error and therefore the effects of administration mode. This difference was the only important one necessary for the Web version because both instruments are self-administered. In fact, to ensure comparability between the paper and Web responses, the formats and wordings of the questions and response categories for the two versions remain identical.
Section 508 of the Rehabilitation Act of 1973, as amended, specifies that persons with disabilities shall have access to and use of the same information that persons without disabilities have. To comply with this section, the O*NET Data Collection Program designed its DOL-sponsored Web applications, including O*NET OnLine, My Next Move, My Next Move for Veterans, Mi Próximo Paso, and O*NET Code Connector, to ensure that the data and information are accessible to the widest possible audience, including people with disabilities. The sites also provide links to several accommodation and disability resources on the Internet. Site accessibility remains an important design component in the ongoing maintenance and development of DOL-sponsored sites, with close adherence to the guidelines of the Web Accessibility Initiative (WAI) from the World Wide Web Consortium (W3C). Web questionnaires are also 508 compliant.
The O*NET Program uses the Internet to gather additional occupational information, such as alternate (lay) titles and high-demand technologies. This use expands the O*NET database, providing easily maintained current information and enhancing users’ ability to find occupations relevant to their training and expertise by giving them a wider range of search terms. This enhancement is accomplished without additional burden to the public and at less cost than other means of data collection.
“Web scraping” is employed to gather information on tools and technologies, with particular emphasis on the technologies that are most in demand by employers. Information from this data mining approach is used to compile a list of “hot technologies.” This list, which is updated quarterly, includes programming and software technologies such as Python and Meditech that appear frequently in lists of occupational or job requirements. This technique is also used to evaluate and augment the lists of tools and technologies developed and maintained for each occupation in the SOC taxonomy. Tools and technologies linked to specific occupations are found by using job advertisement data mining software and by searching job posting, career education, professional association, and other Web sites. Occupational analysts link the tools and technologies to the United Nations Standard Products and Services Code, an online classification system for tools and services (http://www.unspsc.org/). Before the information is published in O*NET OnLine, rigorous reviews are performed to ensure the quality and usefulness of the data. Information collected from the Web in this way represents a significant enhancement to the data available to O*NET users, and it ensures that real-time labor market information is being used to maintain currency of the O*NET database.
A multimethod data collection approach is used to populate the O*NET alternate titles. Data sources include incumbent/occupational expert data, employer job postings, Occupational Code Assignment submissions, transactional analyses (e.g., unmatched search terms submitted to CareerOneStop.org, a DOL-sponsored online resource for job seekers), customer and professional group requests, additional classification systems, and other miscellaneous sources. Each title undergoes an extensive multistep review process and is reviewed by multiple occupational analysts. Deliverables include (1) reported job titles that appear in O*NET OnLine, My Next Move, and Web Services and (2) a published Alternate Titles database that includes alternate job titles appearing in the downloadable database, keyword search, and Web Services. Alternate titles greatly enhance the keyword search functions in the O*NET Web sites. Alternate titles are also incorporated into a number of public and private keyword searches through the O*NET Web Services.
Use of Internet Web sites further enhances the O*NET database, provides greater search capabilities to users, and reduces burden. It also allows for rapid update of the data through user input and at minimal cost. Additionally, this also provides greater accessibility to information for POCs or selected participants, including instances where individuals are unable or unwilling to receive hard copy materials.
To avoid duplication and
reduce cost, several portions of the O*NET Content Model are provided
from existing data sources. Specifically, as described in Appendix B,
the domain of Workforce Characteristics—including information
on industries, job opportunities, and pay—is obtained through
links to existing labor market information databases. Information
about occupational licensing, certifications, related instructional
programs, and associations is provided from existing sources and
several Web sites, including the CareerOneStop (COS) Certification
Finder at
https://www.careeronestop.org/toolkit/training/find-certifications.aspx,
the COS Licensed Occupations Database at
https://www.careeronestop.org/Toolkit/Training/find-licenses.aspx,
and the COS Professional Associations page at
https://www.careeronestop.org/
Developers/Data/professional-associations.aspx.
Wage and employment projections information is provided through work
conducted by the BLS, including by the OES program
(https://www.bls.gov/oes/)
and the Employment Projections program (https://www.bls.gov/emp/).
The exhaustive reviews of existing labor market and occupational information conducted by the DOL’s review staff, as well as subsequent research, identified no other comprehensive, valid, and reliable sources that could be used for the data items included in the O*NET database. The development of the O*NET Program has involved staff and advisors who have many years of experience in labor market and occupational information and who are familiar with existing data sources. In fact, as discussed in Section 2, many existing systems that provide detailed occupational information are actually using information based on O*NET data or the predecessor DOT.
The few existing sources with similar measures are too limited to be used in the O*NET database. Some existing sources are valid and reliable—for example, information from the U.S. Office of Personnel Management and the U.S. Department of Defense—but are not comprehensive, because they represent only those jobs in federal civilian employment or the military. Some private sources of job analysis information exist; however, they are based on job analyses conducted for particular purposes or settings rather than on a representative sample of employers and workers. They are therefore limited in their coverage and not representative of the entire workforce. Furthermore, these analyses are not comparable because they do not use the prescribed O*NET common language to describe occupational requirements; it is not practical to combine them, because they include dissimilar components. Finally, these private data sources are not available to the general public.
The selected samples for most occupations represent all sizes of establishments. Occasionally, the targeting strategy used in selecting an efficient sample may lead us to omit some small establishments from the sampling frame, but this omission occurs for few occupations. The omission is allowed only when it is clear that sampling small establishments will greatly reduce the efficiency of the data collection or that incumbents from small establishments are not working in the mainstream of the occupation.
Because establishments of all sizes should be represented in the samples for most occupations, specific design provisions have been undertaken to avoid overly burdening small establishments. For example, Exhibit A-1 shows the distribution of establishments by number of employees on the D&B list used as the sampling frame for O*NET data. It illustrates how O*NET sampling selects small establishments at a much lower rate than that at which they occur in the population. For example, although 95.2% of establishments employ fewer than 25 employees (represented in the first three rows of the exhibit), only 45.7% of the O*NET sample will consist of such small establishments. On the other hand, large establishments (with 250 or more employees, as represented in the 7th to 9th rows of the exhibit) will make up 19.0% of the O*NET sample but only 0.7% of all establishments. Thus, to reduce the burden on small establishments with few employees, the O*NET sample relies more heavily on large establishments. The disproportionate sampling of large and small establishments is properly accounted for in the analysis weighting, resulting in statistically consistent estimates.
In addition, data collection procedures place lower burden on small establishments than on large establishments. When a small establishment is selected, it likely employs fewer of the targeted occupations and has fewer employees working in the occupations. Thus, a POC at a small establishment generally spends less time preparing sampling lists and distributing questionnaires than a POC at a large establishment, which is more likely to employ several of the targeted occupations and have a large number of employees working in the occupations.
Exhibit A-1. Distribution of Frame and Sample Establishments by Employment Size
Number of Employees |
Total Number of Frame Establishmentsa |
Frame Distribution (Percent) |
Actual Distribution of O*NET Sampled Establishmentsb (Percent) |
1–4 |
13,716,903 |
75.1 |
19.1 |
5–9 |
2,388,040 |
13.1 |
6.9 |
10–24 |
1,283,308 |
7.0 |
19.7 |
25–49 |
391,922 |
2.1 |
8.5 |
50–99 |
230,493 |
1.3 |
16.5 |
100–249 |
123,539 |
0.7 |
10.3 |
250–499 |
32,738 |
0.2 |
10.9 |
500–999 |
13,284 |
0.1 |
4.7 |
1,000+ |
8,908 |
<0.1 |
3.4 |
Unknown |
75,124 |
0.4 |
0.0 |
Total |
18,264,259 |
100% |
100% |
Note: Percentages may not sum to 100% because of rounding.
a Data based on June 2023 Dun & Bradstreet (D&B) frame of establishments.
b Data based on distribution of O*NET samples completed over the last five years that used the D&B frame. Future O*NET samples will be similarly designed.
The O*NET database is the most comprehensive source of occupational information in the United States. No other similarly comprehensive, reliable, and valid source exists. If O*NET data are not collected, U.S. citizens, industry, business establishments, military, government and educational institutions, and the workforce investment system will have few options that meet their needs for descriptive occupational information. O*NET data are used to develop industry competency models and occupational competency profiles (i.e., industry- or occupation-specific job or work analyses). O*NET data also include information on transferable skills and are used for skills gap analysis, promoting the development of a mobile workforce responsive to changing economic needs. “Students, jobseekers and workers need up-to-date information on required job skills for specific occupations. O*NET provides the best resource for detailed descriptions of the knowledge, skills, abilities, work-related tasks, and tools and technologies used by specific occupations (974 occupations are covered)” (Workforce Information Advisory Council, 2017).
The use of O*NET data by industry, employers, software developers, job seekers, students, educators, and workforce development specialists supports a well-functioning U.S. labor market and workforce investment system—both essential to U.S. competitiveness in the global 21st-century economy. O*NET-SOC occupations conform to the SOC, permitting O*NET data to be linked to and analyzed with sources of information on current occupational employment and trends, wages, and demographic data. Its electronic format is freely accessible through DOL-sponsored Web sites, including O*NET OnLine, My Next Move, My Next Move for Veterans, Mi Próximo Paso, and O*NET Code Connector. A wide variety of database versions are also available to download free of charge from the O*NET Resource Center (see https://www.onetcenter.org/). Over 3,800 customers are signed up for O*NET Web Services, including private, nonprofit, government, and military organizations, allowing them to take advantage of the ability to access and seamlessly incorporate O*NET data and applications within their systems via the extensive Web Service offerings (see https://services.onetcenter.org). The O*NET data and structure are also being incorporated into a number of open data initiatives, including the Data at Work initiative, OpenGovernmentdata.org, the Credential Engine/Credential Registry, and the Competency and Skills System.
The initial 3.1 version of the O*NET database has been updated 20 times as new data have been collected and analyzed. Additional data releases are planned through 2024 allowing for the continued update of occupations and release of data on new and emerging occupations. The consequences of discontinuing data collection would be that the millions of users who rely on O*NET data for business and career decisions, for educational programming, and for work in human resources or workforce development would instead be using portions of information that are out of date and incomplete. If data were collected less frequently, the currency of some data would become questionable, especially for occupations that are changing as a result of new technologies. The focus of data collection on high-growth and new and emerging occupations reflects the need to provide current information in a rapidly changing, demand-driven economy.
A 3-year extension of the O*NET Data Collection Program is being requested for the period of March 2024 through February 2027. This extension will provide for the updating of selected high-growth occupations and for data collection activities for new and emerging occupations. The 2018 Standard Occupation Classification system identified many new occupations (such as data scientists), as well as a number of modifications to existing occupation titles, definitions, and classifications (https://www.bls.gov/soc/2018/home.htm). A dynamic and progressive U.S. economy requires continuous improvement to the data on which so many decisions are based. Millions of people are currently using O*NET information, and the numbers continue to expand as public agencies and private developers integrate O*NET data into their systems and products. The O*NET database provides valid, reliable, and current occupational information crucial to a strong U.S. workforce. O*NET database updates are scheduled to occur once a year to incorporate newly collected information on recently surveyed occupations. A schedule for data analysis is provided in Exhibit A-5 in Section 16; schedules for data collection and analysis are subject to annual appropriations.
Requiring respondents to report information to the agency more often than quarterly;
Requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;
Requiring respondents to submit more than an original and two copies of any document;
Requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records, for more than three years;
In connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;
Requiring the use of a statistical data classification that has not been reviewed and approved by OMB;
That includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or
Requiring respondents to submit proprietary, trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.
The ICR requires no special circumstances nor record keeping requirements.
In accordance with the Paperwork Reduction Act of 1995, the public was provided with 60 days to comment through a Notice published in the Federal Register on July 31, 2023, 88 FR 49502.
The Department of Labor received a single comment in response to the 60-day FRN, which contained two suggestions. One was to update the dates for the requested renewal period from that contained in the draft, which has been done. The other was a suggestion to include a reference to language from the President’s FY 2024 Budget Request in the section describing the fund source that includes resources for the O*NET system. In its stead, the Department has included cost estimates for operation of the O*NET program in section 14 of this Supporting Statement A, and in Exhibit A-3. Annualized Costs to the Federal Government.
The 2021 OMB clearance package was reviewed by an expert consultant, Dr. James B. Rounds (only increased use of the Occupation Expert Method occurred between the prior and current 2024 OMB packages). Dr. Rounds is a professor in the Psychology Department at the University of Illinois at Urbana-Champaign. He is a widely published industrial/organizational psychologist who is intimately familiar with the O*NET program and the types and uses of the data collected. Revisions responsive to his comments have been incorporated, as appropriate, in the clearance package. Contact information for Dr. Rounds is: (217) 244-7563 and [email protected].
The data collection contractor, RTI International, has several mechanisms in place to obtain ongoing feedback from study participants. The BLs have multiple telephone contacts with POCs and occupation experts; they are careful to document in their call notes any significant comments or suggestions they receive. Subject matter experts at the professional associations that we contact to help identify occupation experts frequently provide feedback that the National Center for O*NET Development uses to refine occupation descriptions and tasks. Letters and brochures sent to POC’s, and occupation experts offer a toll-free number they can call with questions or comments. In addition, the project Web site, https://onet.rti.org/, which both study participants and the public can access, has a “contact us” tab that offers both a toll-free number and an e-mail link. Finally, both the paper and online versions of the questionnaires provide respondents an opportunity to submit comments with their questionnaire responses. All feedback received from these sources is promptly reviewed by project management staff. Of course, the agency will comply with all Paperwork Reduction Act requirements should comments warrant changing the information collection.
Since the origins of the survey, incentives have been offered to POCs, establishments, and employees to encourage their participation in the O*NET Data Collection Program. Although the procedures are designed to minimize respondent burden, the effort for the company and the POC participation is not insignificant.
The POC’s responsibilities include:
reading the introductory package to become familiar with the purpose of the O*NET Data Collection Program and the role of a POC;
seeking permission within the company, as necessary, to participate in the O*NET Data Collection Program;
making a roster of all employees at the location who work in as many as five different occupations;
participating in a sampling process that selects as many as 20 total employees from these occupations, and maintaining this sample roster for future reference;
distributing questionnaires to the sampled persons within the company and addressing their questions and concerns about the survey; and
distributing follow-up materials to employees, including thank you/reminder cards and replacement questionnaires, and following up with nonrespondents to encourage participation.
Because POCs are the only link with the O*NET respondents, they must be fully committed to the data collection process. The POC is the O*NET Program’s representative within the establishment who communicates the importance of the O*NET Program. The employer is also essential because he or she is being asked to:
support the O*NET Data Collection Program by agreeing to the company’s participation in data collection,
support and encourage the POC in carrying out his or her responsibilities,
allow the POC to provide information regarding the number of persons employed in the establishment in the occupations of interest, and
support the participation of the sampled employees.
Incentives for both POCs and employers are essential to encourage the highest response rates possible. POCs who agree to participate receive a Certificate of Appreciation from DOL. The Certificate of Appreciation is printed in color on card stock, bears the DOL Seal and O*NET logo, displays the POC’s name, and is signed by a high-ranking DOL official.
Employers who agree to participate receive a link to the O*NET Toolkit for Business. The toolkit is an O*NET Program information packet, including a guide for writing job descriptions, that managers can use for human resource planning.
The continuation of these incentives is planned for both the POC and the employer because they seem to be working quite well, as evidenced by O*NET’s competitive employer response rate (see Appendix D). An earlier experiment was conducted from 2002 to 2004 to evaluate an additional incentive of $20 to the POC. At that time, the incentive had no effect on POC cooperation rates or employee response rates, but it significantly increased data collection costs. These experimental findings suggested that the POC and employer incentives were adequate for maximizing response rates at a reduced cost (Biemer et al., 2006).10 As noted in Supporting Statement Part B, Section 4.11.2 and Appendix I, we are proposing to revisit the idea of offering a monetary incentive to participating POCs who complete recruiting and sampling activities to assess the potential impact for reducing nonresponse bias, increasing response rates, and minimizing follow-up data collection efforts.
Since 2001, each employee is offered a prepaid incentive of $10 (cash or digital gift-card) to ensure that a high percentage of the job incumbents respond by completing the questionnaire. This initial amount was based on evaluating numerous payment amounts in the pretest to determine the optimal means to maximize the response rate. As demonstrated in Appendix D, O*NET, like the broader field of survey work, has been impacted by a gradual decline in response rates over the past several years. To further reduce the potential for nonresponse bias, maximize response rates and decrease any follow-up data collection efforts, an experiment for assessing the impact of offering a post survey completion incentive is being proposed (See Supporting Statement Part B, Section 4.11.2 and Appendix I). The proposed amounts were based on continual survey respondent feedback regarding the level of effort required to complete the survey as well as an assessment of the impact of differing amounts in the current economic market. Offering an additional post survey completion incentive was chosen over either increasing the pre-incentive or just offering a post incentive to both minimize additional costs and to maximize participation (i.e., the pre-incentive helps to establish the study’s legitimacy while the post-incentive only offers a payment for services completed, and this pre/post design with larger incentive amounts produced the highest response rate in the pretest incentive experiment). Studies have supported the use of pre and post incentives together to improve response rates (Dillman, 2020; Asire, 2017). It will also be noted to the selected participant, that completion via the web will result in an expedited receipt of their post-incentive (digital gift-card). Any increase in web participation helps to further reduce costs by excluding return shipping and data entry costs as well as increases data quality by reducing response errors (e.g., multiple responses, non-valid responses).
Monetary incentives have the greatest potential impact when the respondent must exert some special effort, such as taking a test or filling out a multi-item questionnaire. The incentive encourages respondents in a task requiring higher levels of involvement and commitment than the typical one-time, face-to-face interview. Although the O*NET questionnaires are not tests, the cognitive demands they place on respondents resemble test-taking demands in that the respondents must assess the requirements of their jobs. These monetary incentives are instrumental in impressing upon the respondent the importance of this rating task. Respondents who perceive the rating task as important will likely make thoughtful, carefully considered assessments rather than hasty ones, thus improving the reliability of the data.
In addition, the monetary incentives are important because respondents are encouraged to complete the questionnaire, which takes about 30 minutes, on their own time rather than on the job. This encouragement minimizes the burden on employers and also improves the quality of the data; without it, busy workers might be underrepresented in the data, which would bias the estimates for job performance.
These monetary incentives may at least partially offset its inherent cost through efficiencies created in the data collection process as a result of higher response rates (Statistics Research Division, 2000). For the job incumbent respondents especially—although they are not viewed as a difficult-to-reach population in the usual sense—considerable effort and resources are expended to identify and reach them through the sampling process. Each one represents a worker in a specific occupation in a specific establishment in a specific industry. The expense of reaching that particular respondent justifies the cost of a monetary payment to ensure a high rate of response.
Occupation experts provide data for approximately 25% of the O*NET-SOC occupations, with an expected increase to 35% by November 2024. Each occupation expert who agrees to participate receives a prepaid incentive of $40 (cash or digital gift-card) and a Certificate of Appreciation from DOL. Unlike job incumbents, who complete only one domain questionnaire, occupation experts are asked to complete all three domain questionnaires; thus, the $40 incentive is about $13.33 per questionnaire. This incentive is slightly higher than the $10 offered to Establishment Method respondents for completing one domain questionnaire. The increased pre-paid incentive and the Certificate of Appreciation are necessary to gain cooperation from what is often a rare group of experts for an occupation.11 Moreover, the additional incentives seem commensurate with the effort involved in responding to multiple questionnaires, given that occupation experts are supervisors and trainers in the occupation and, as such, earn a higher salary than the average employee.
As noted above, selected OE respondents are offered a higher incentive amount than employees based on differing levels of effort. By assumption, if employee respondents are offered a maximum of $30 in total from their respective incentive experiment and this produced a successful outcome, then OE respondents should have their incentive amounts adjusted accordingly. However, tripling the $40 to a maximum of $120 seems unfeasible especially given the higher response rate among OEs where we have direct contact to solicit participation. Therefore, an experiment for assessing the impact on response rates and nonresponse bias from varying amounts of a post-completion incentive is being proposed (See Supporting Statement Part B, Section 4.11.2 and Appendix I) to ensure equity between the Establishment and Occupation Expert methodologies.
O*NET Online currently includes Sources of Additional Information (SAI). These are links to associations related to each occupation in O*NET. The SAI is available to O*NET career explorers and entrants, so they have the most current information when exploring the world of work. The program also publishes a list of professional associations that have endorsed the O*NET data collection. To continue to provide recognition to organizations that participate and endorse the O*NET data collection, and to promote participation among potential SOs for OE data collection, during the summer of 2022, the O*NET project fielded a recognition program known as the Ally Campaign. During this time outreach was made to National and Regional SOs that had previously participated in the OE data collection since January 2019. An SO was considered participating if they provided a listing of OEs or if they facilitated a volunteer request. Participating SOs were contacted via email and offered an O*NET Ally designation. Over 100 SOs became O*NET Allies. The campaign was so successful that offering it during active data collection would be beneficial not only to the SOs but also increase participation of potential SOs.
The O*NET OE SO recognition program will be implemented during the recruitment of SOs for participation. During that time National and Regional organizations will be offered the O*NET Ally designation in exchange for their participation. Benefits of being on O*NET Ally include:
Addition of the organization to the Sources of Additional Information (SAI) in O*NET OnLine
An exclusive O*NET Ally tag added next to the SO name on the SAI.
Addition to the O*NET Ally Listing in a special section of the O*NET Resource Center.
Access to the O*NET Ally Welcome page to download the O*NET Ally badge to use within their communication material.
A National and Regional SO can participate in the OE data collection without accepting the O*NET Ally designation.
Respondents are informed that their individual responses will be kept private to the extent permitted by law. A system privacy statement is located on the system website at https://www.onetonline.org/help/privacy/. Survey data are collected from job incumbents (Establishment Method) and from occupation experts (OE Method). Informational materials provided to potential respondents contain essential program information and assurances of privacy that enable the person to make an informed decision about his or her voluntary participation in the data collection effort. Examples of informational materials provided to survey participants, as well as a listing of enhancements appear in Appendix E.
Employees sampled at establishments are asked to complete their questionnaires on their personal time, not company time. This stipulation enables the employee to select a comfortable and private setting, if desired, in which to complete the questionnaire. All respondents have a choice of completing paper questionnaires or completing the questionnaires online at the project Web site. Paper questionnaires are mailed directly to RTI in a stamped reply envelope provided by RTI. The individual responses are processed according to a study ID number. All O*NET Data Collection Program staff are required to sign a privacy pledge that assures each respondent that the privacy of responses to the questionnaire will be maintained. Only authorized staff have access to the completed instruments and data files. The completed and processed questionnaires are stored in a secure document-control area until federal authorization has been granted to destroy them. All computer files, including those associated with the control system, are password protected.
The Internet-based system that allows respondents to provide their survey responses electronically has restricted access, including a user ID and password authentication protocol for respondents. The Web server includes an SSL certificate, the same technology used by e‑commerce sites, to allow encrypted transmission of all information over the Internet. The database containing the survey data is not accessible via the Internet; it resides on a server inside the RTI firewall. Only O*NET Data Collection Program staff have access to the master survey database.
The O*NET questionnaires (see Appendix F) collect very little personal information about the respondent, and what is collected contains no identifiers, such as personal name or place of employment. No individual-level data are published, nor are they accessible or provided to anyone except the O*NET Data Collection Program staff. Published results are made available only in aggregate, as one set of estimates for an entire occupation. Furthermore, no demographic data (e.g., sex, race) are released, even in aggregate form. Finally, estimates are not produced for any subpopulations within an occupation, such as geographic region or sociodemographic group, as these details could allow the identification of individuals.
Before publishing the O*NET tables on the public Web site, the O*NET Program team thoroughly examines the tables for any risk of disclosure of private information. In particular, each table is analyzed to identify any “sensitive” cells (i.e., cells that may reveal too much information about an individual employee). Willenborg and De Waal (1996) have recommended using an (n,k)-dominance rule that a cell be regarded as sensitive if the sum of the largest n contributions account for more than k% of the total cell value. Willenborg and De Waal further recommend that n = 5 and k = 80. Because every sampled employee contributes only one response, these recommendations translate into a minimum cell size of 5/0.80, or about 6. In fact, the minimum cell size for the O*NET tables is 10 respondents; any sensitive cells with fewer than 10 are suppressed. In addition, the (n,k)-dominance rule assumes a complete census. As Willenborg and De Waal note, when applied to tables based on samples and where the cell entries are weighted averages, the (5,0.80) rule affords even greater disclosure control. This extra control ensures that these O*NET tabular data pose no disclosure risks to any individual respondent.
Only three questions in the O*NET questionnaires may be considered to be of a sensitive nature. In the Background Questionnaire, the survey uses the three disability questions developed for the Census Bureau’s American Community Survey. The first two questions ask respondents about serious hearing and sight difficulties. The next question has subparts that ask about difficulties with concentration, memory, or decision making; mobility; self-care, and whether respondents’ physical or mental health makes it difficult to do errands alone. Completion rates for these items indicate that the great majority of participants (97%) elect to answer them.
The O*NET sampling strategy is to randomly select participants at the individual level. The disability questions, together with the demographic questions, provide descriptive information about the sample of respondents.
Respondents to the O*NET survey are informed that responding to all questions, which includes disability status and the other demographic characteristics, is voluntary and that the data will be kept private to the extent permitted by law. They complete the survey on their own time, in a private setting if they choose. No identifying information, such as the respondent’s name or place of employment, is recorded on the questionnaire.
Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.
If this request for approval covers more than one form, provide separate hour burden estimates for each form.
Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included in Item 14.
As described in Section 3.1, there are two methods for O*NET data collection: the Establishment Method and the OE Method. The Establishment Method uses a two-stage sample, with establishments selected at the first stage and employees at the second stage. Thus, there are burden hours associated with both establishments and employees. For the OE Method, burden is incurred only by sampled occupational experts who return questionnaires. This submission also includes burden to complete a focus group of up to 30 respondents per year; these focus groups are intended to solicit feedback on data collection protocols and/or materials, as needed, to inform enhancements to O*NET data collection in future years (See Supporting Statement Part B, Section 4.11.3).
Exhibit A-2 presents the estimated annualized cost and hour burden across participating establishments and individuals. The burden estimates are driven by the average annual number of respondents and the average time per response. The eligibility and participation rates assumed to calculate the average annual number of respondents are based on prior years of O*NET data collection (reflecting conditions since the onset of COVID-19) and assume that up to 100 occupations will be completed annually: 60 via the OE Method and 40 via the Establishment Method.12 For more details on the calculation of the burden estimates, see Appendix G.
The time per response for a sampled establishment depends on the number of activities completed among those listed below. Most establishments do not complete all activities because of ineligibility and nonresponse. A POC is identified within participating establishments to coordinate the following data collection activities:
Verification call to confirm the establishment is in business (2 minutes);
Screening call to identify whether any occupations of interest are present (3 minutes);
Recruiting call to gain participation in O*NET (12 minutes);
Creating lists of employees in the selected occupations for sampling (20 minutes);
Sampling call to select employees for the survey (10 minutes);
Distribution of questionnaire packets (15 minutes); and
Up to four follow-up calls to POC (2 minutes per call).
The total time per response for an establishment can range from 2 to 70 minutes, depending on how many of these activities are completed. Past years of O*NET data collection experience show that the average time per response is 15.8 minutes, or 0.263 hours.
The average time per response for sampled individuals is 30 minutes per domain questionnaire. Employees sampled under the Establishment Method are asked to complete one domain questionnaire (0.5 hour per response), and occupational experts sampled under the OE Method are asked to complete three domain questionnaires (1.5 hours per response). It is assumed that each focus group respondent will incur up to 120 minutes of burden.
All O*NET activities are voluntary and contribute only reporting or processing burden. No record-keeping burden is incurred.
Exhibit A-2. Estimated Annualized Cost and Hour Burden
Activity |
Number of Respondents |
Number of Responses per Respondent |
Total |
Average Burden Hours |
Total Annual Burden Hours (Rounded) |
Hourly Ratea |
Total Burden Cost |
Individuals |
|
|
|
|
|
|
|
[1] Establishment Method employee respondents |
6,063 |
1 |
6,063 |
0.500 |
3,032 |
46.27 |
$140,291 |
[2] Occupation Expert Method respondents |
2,025 |
1 |
2,025 |
1.500 |
3,038 |
46.27 |
$140,569 |
[3] Focus Group respondents |
30 |
1 |
30 |
2.000 |
60 |
46.27 |
$2,777 |
Unduplicated Totals |
8,118 |
-- |
8,118 |
-- |
6,130 |
-- |
$283,637 |
Establishments |
|
|
|
|
|
|
|
[4] State/local/tribal governments |
974 |
1 |
974 |
0.263 |
256 |
76.18 |
$19,503 |
[5] Federal government |
1,619 |
1 |
1,619 |
0.263 |
426 |
76.18 |
$32,453 |
[6] Private sector |
31,704 |
1 |
31,704 |
0.263 |
8,338 |
76.18 |
$635,189 |
Unduplicated Totals |
34,297 |
-- |
34,297 |
-- |
9,020 |
-- |
$687,145 |
Grand Totals |
42,415 |
-- |
42,415 |
-- |
15,150 |
-- |
$970,782 |
a The average annual hourly rate was estimated using the December 2022 Employer Costs for Employee Compensation Summary issued by BLS on March 23, 2023. The average total compensation per hour for private industry was $40.23, which was inflated based on the Employment Cost Index to a median hourly total compensation of $43.99 for March 2024–February 2025, $46.23 for March 2025–February 2026, and $48.59 for March 2026–February 2027. The average of the three inflation-adjusted rates ($46.27) is used for estimating the employee/individual cost burden. Given that the establishment POC will often be a human resources manager, the December 2022 total compensation rate of $67.64 for the Management, Professional and Related category was inflated to a median hourly total compensation of $72.95 for March 2024–February 2025, $76.14 for March 2025–February 2026, and $79.46 for March 2026–February 2027. The average of the three inflation-adjusted rates ($76.18) is used for estimating the establishment cost burden.
Source: Bureau of Labor Statistics, U.S. Department of Labor. (2023, March). Employer costs for employee compensation summary. Economic News Releases. https://www.bls.gov/news.release/pdf/ecec.pdf
The estimated annual cost to the government for the O*NET Data Collection Program for the period March 2024 through February 2027 is approximately $9.7 million. This estimate includes all direct and indirect costs of conducting the sampling, data collection, and analysis activities of the O*NET Data Collection Program. In Exhibit A-3, personnel and fringe benefit costs are for the grantee (N.C. Department of Commerce) personnel who manage the O*NET Data Collection Program. The grantee subcontracts certain program activities (e.g., survey operations and data analysis); those costs are listed on the contractual line.
Exhibit A-3. Annualized Costs to the Federal Government
Item |
Amount |
|
|
Personnel |
$532,794 |
Fringe benefits |
$207,598 |
Travel |
$30,000 |
Equipment |
$4,000 |
Supplies |
$41,400 |
Contractual |
$8,529,000 |
Costs for incentives |
$239,315 |
Total direct costs |
$9,584,107 |
Indirect charges |
$127,604 |
Grand Total |
$9,711,711 |
As described in earlier sections, incentive experiments are being proposed at the establishment, employer and occupation expert levels, and contacted Occupation Expert source organizations are presented with the opportunity of a recognition program for their participation (Section 9). Informational materials have been condensed to reduce redundancy and burden (Appendix E) and the look and format of the questionnaires have been modernized to reflect current best practices for surveys, including standardization of similar background questions between the Establishment and Occupation Expert questionnaires; and both the instructions for, and the presentation of level scale anchors have been updated to lend additional clarity (Appendix F). The Department does not believe that these changes in requirements will impact the burden calculation because the burden is already accounted for as an average or these activities are already subsumed within the current protocols. However, the increased use of OE Method to improve overall data collection efficiency is one contributing factor to offset this burden increase and result in an overall decrease in the estimated burden, as described below.
Exhibit A-4 compares the burden projected for the March 2024–February 2027 period with the burden requested for the previous 3-year period, December 2021–November 2024. Estimates for the previous 3-year period are available from the OMB Supporting Statement, Part A, dated November 2021. The average annual burden hours projected for March 2024–February 2027 are 15,150, which represents a decrease from the previous 3-year period (2021–2024), when 16,446 hours were requested. The annual average number of responses for the March 2024–February 2027 period is 42,415, whereas 40,942 responses were estimated for the 2021–2024 period (U.S. Department of Labor 2021).
The increase in the annual average number of responses is primarily due to an increase in both the Establishment and OE sample sizes, as designed to mitigate a continued decline in participation for the Establishment Method. The AAPOR, the Census Bureau, and others have widely noted the general downward trend in survey participation (Tourangeau, 2017; Office of Survey Methods Research, 2023; American Association of Public Opinion Research, n.d.). Over the past decade, O*NET has also experienced a gradual decline in response rates, particularly for the Establishment Method (see Appendix D for a summary of response rate experience to date). We further expect that the pandemic has had lasting impacts on participation and eligibility. Given these uncertainties, we have increased the assumed sample size requirements for the 2024–2027 burden period from approximately 3,000 to 4,430 establishments per month. The target number of OE respondents per year has also been increased from 1,000 to 2,025 to capitalize on the efficiency offered by the OE Method. Although the potential increase in the sample sizes appears large, the sample sizes may be smaller in practice. If production is higher than expected for the Establishment Method, O*NET’s Model-Aided Sampling (discussed in Supporting Statement Part B, Section 2.1) paradigm permits early termination of data collection to prevent overproduction and safeguard against burden. Additionally, sample sizes may be limited by availability of appropriate staff to conduct data collection.
Despite an increase in the annual average number of responses, the projected number of burden hours has decreased. The decrease in the estimated burden is due to two factors. First, we have proposed an increased use of the OE Method, which incurs less burden per occupation, to improve overall data collection efficiency. Specifically, the assumed number of OE Method occupations completed per year has been increased from 40 to 60 for the March 2024-February 2027 burden period. Fewer Establishment Method occupations will be targeted per wave of sample accordingly. This adjustment is designed to help ease some of the establishment burden. Like the Establishment Method, the OE Method has experienced some decline in participation but to a lesser extent; this is likely because individuals who are sampled via the OE Method are contacted directly, unlike the Establishment Method, for which selected individuals remain anonymous and questionnaire distribution is coordinated through a POC. Thus, the OE Method provides an efficient approach for collecting and updating occupations that are increasingly difficult to survey under the Establishment Method.
The second factor leading to a decline in the estimated burden is that we have corrected an overestimation in the number of Establishment Method employee respondents. In prior years’ estimates of burden, the assumed number of questionnaires returned per establishment did not fully account for establishments that fail to return at least one questionnaire. This has been corrected for the 2024–2027 burden estimates.
Last, Exhibit A-4 compares the estimated annual average cost to respondents for March 2024–February 2027 with the average annual cost estimated for 2021–2024. The increased annual costs since 2021–2024 are primarily due to an inflation adjustment.
More detailed comparisons of the hour and cost burden between the 2021–2024 and 2024–2027 burden periods may be found in Appendix G.
Exhibit A-4. Comparison of the Average Annual Hour and Cost
Burden
Between the 2021–2024 and 2024–2027 Burden
Periods
|
Annual
Average |
|
Annual
Average |
||||
Activity |
Total |
Total |
Monetized |
|
Total |
Total |
Monetized |
Individuals |
|
|
|
|
|
|
|
[1] Establishment Method employee respondents |
13,068 |
6,534 |
$252,670 |
|
6,063 |
3,032 |
$140,291 |
[2] Occupation Expert Method respondents |
1,000 |
1,500 |
$58,005 |
|
2,025 |
3,038 |
$140,569 |
[3] Focus Group respondents |
0 |
0 |
$0 |
|
30 |
60 |
$2,777 |
Establishments |
|
|
|
|
|
|
|
[4] State/local/tribal governments |
578 |
181 |
$12,670 |
|
974 |
256 |
$19,503 |
[5] Federal government |
1,706 |
534 |
$37,380 |
|
1,619 |
426 |
$32,453 |
[6] Private sector |
24,590 |
7,697 |
$538,790 |
|
31,704 |
8,338 |
$635,189 |
Unduplicated
Totals |
40,942 |
16,446 |
$899,515 |
|
42,415 |
15,150 |
$970,782 |
a From the OMB Supporting Statement, Part A, dated November 2021.
b From Appendix Exhibit G-2.
The major components of the O*NET Data Collection Program include sampling, data collection operations, and analysis. Exhibit A-5 shows the expected schedule for annual data analysis cycles and data publications for the next few years. Ranges for the estimated number of published occupations presented for Analysis Cycles 26-28 demonstrate the uncertainty of the impacts of COVID-19 on participation, and declining response and business eligibility rates. Projections are based on available funding to conduct data collection.
Exhibit A-5. Data Analysis and Publication Schedule
Analysis Cycle |
Analysis Cycle Start Date |
Analysis Cycle End Date |
Estimated Establishment Method O*NET-SOC Occupations |
Estimated Occupation Expert Method O*NET-SOC Occupations |
Estimated O*NET-SOC Occupations Published |
Publication Date |
26 |
July 2024 |
July 2025 |
20–40 |
30–60 |
50–100 |
July 2025 |
27 |
July 2025 |
July 2026 |
20–40 |
30–60 |
50–100 |
July 2026 |
28 |
July 2026 |
July 2027 |
20-40 |
30-60 |
50-100 |
July 2027 |
Described here is the approach used for data cleaning and editing, as well as the analyses that are performed annually.
Paper questionnaires are manually and machine edited so that completely blank questionnaires are removed; responses to items that should have been skipped are blanked out; multiple responses are blanked out; and codes indicating missing data, multiple responses, and legitimate skips are inserted. Codes for legitimate skips and missing responses are also inserted in the records for questionnaires obtained through the Web. In addition, an electronic check is conducted to detect duplicate questionnaires from the same respondent.
Analyses of the questionnaire ratings are based on the assumption that raters were qualified, willing, and able to engage in the rating tasks. This task is accomplished by requiring each case to first pass through a series of machine edits using prescribed eligibility criteria, including having at least one task rated important and having at least 50% of the domain questionnaire items completed. Cases not meeting these criteria are excluded from the analysis file. Cases with certain questionable characteristics are flagged in this editing process, and analysts review these cases to determine their completion status. Flagged for review are all cases in which the respondent (1) indicated in the “global match” item that the target O*NET-SOC occupation description did not at all describe his or her own job and (2) rated fewer than one third of the tasks as important. An analyst reviews the self-reported job titles of these cases to determine if they are at all likely to belong in the O*NET-SOC occupation. If a case does not belong, it is removed from further analysis. If, in the judgment of the analyst, there is a reasonable chance that the case belongs in the O*NET-SOC occupation, it is sent to the next stage of review.
For a subset of O*NET-SOC occupations, the self-reported job titles are also reviewed by an analyst to identify potential “proxy respondents,” or cases in which someone other than the sampled person responded to the questionnaire on the incumbent’s behalf. For example, a nurse may respond to the survey on behalf of a selected doctor. This review is performed for a subset of O*NET-SOC occupations that were previously identified as having a higher-risk of proxy respondents. Proxy respondents were detected primarily when their own self-reported job title (e.g., “Office Manager”) differed from that of the person for whom they were responding (e.g., “Surgeon”).
Finally, cases that pass the machine edits and the analyst review are subjected to a deviance analysis designed to identify cases that are outliers relative to other cases in their occupation. The deviance analysis involves two procedures: (1) a statistical procedure to quantitatively identify potential outliers and (2) an analyst review of these potential outliers to make the final decision for each case. Cases that do not pass the analyst review are deemed deviant within their occupation and are removed from the data set. On average, these activities cumulatively eliminate about 10% of all returned questionnaires. The cases passing all data cleaning criteria are used to create the estimates for publication.
Basic sampling weights are applied to the data to make inferences about the population of incumbents for each occupation.13 These weights are computed as the inverses of the overall selection frequencies and the selection probabilities for each selected establishment and each individual participant. The analysis weights for the eligible sample units are adjusted to compensate for unit nonresponse for both establishments and employees, multiple subwaves of sampling, and sample adjustments. In addition, when variation in the weights is large, the weights are trimmed to reduce the variation. To maximize comparability of O*NET estimates to estimates from other federal sources, the final sample weights are also ratio adjusted to occupation estimates obtained from the OEWS survey.
Sampling errors are computed. The analysis weights used in the sampling error computations, as noted, have been adjusted for nonresponse and are consistent with the complex sampling design.
For each occupation, the sample size, mean, standard deviation, and standard error of the ratings for each descriptor are calculated, together with the 95% confidence interval around the mean. Estimates with questionable precision are flagged and recommended for suppression if any of the following conditions is true:
The sample size (i.e., number of respondents who answered the question) is less than 10.
The variance is zero and the sample size is less than 15.
The relative standard error (RSE) is greater than 50%.14
On average, 3–4% of the estimates are flagged or suppressed.15
Interrater reliability and agreement are assessed with three different analyses. The first two measure reliability in terms of the covariation among ratings, and the third reflects rater agreement. The intraclass correlation coefficients (ICCs) for each questionnaire item across all occupations are computed. These results allow one to compare respondent rating variance within an occupation with respondent rating variance across occupations. In addition to the ICCs, the mean interrater correlations (Pearson’s r) are calculated for all pairs of raters within each occupation. Finally, to assess the absolute difference among ratings of each item within each occupation, an average deviation index is calculated for each item within each occupation. For any given item and occupation, the average deviation index measures the average extent to which each individual rating deviates from the item mean. Some differences in ratings within occupations are expected because O*NET-SOC occupations comprise a variety of different jobs in most cases.
Each of the reliability analyses conducted (rater, standard errors) is influenced by the number of respondents. The O*NET data collection methods include a sufficient number of respondents in each occupation to ensure reliability (Peterson et al., 1997b).
The O*NET database is scheduled to be updated annually. Each update will include data for those occupations collected and analyzed during the previous 12-month period. Consequently, a database update includes occupations from multiple data collection waves, depending on the number of prior waves for which analysis was completed that year. For each occupation collected, the newly calculated means data replace existing analyst-based or incumbent-based means data in the database. Users are provided with metadata regarding when the data were collected and other pertinent information that will assist the users in interpreting the data.
The O*NET database is designated with a version number denoting each update (e.g., from O*NET 27.0 to O*NET 27.1). The database is developed and administered with the MariaDB database management system. Once the MariaDB database is updated, it is used to generate the database for public release as a series of database formats, including MYSQL, SQL Server and Oracle. It is accessible to the public on the O*NET Program Web site at https://www.onetcenter.org/.
The expiration date will be displayed on the cover of the survey questionnaires.
There are no exceptions.
B. COLLECTIONS OF INFORMATON EMPLOYING STATISTICAL METHODS.
This collection employes statistical methods.
1 More information about the O*NET Data Collection Program can be found at the National O*NET Program’s public Web site, https://www.onetcenter.org/ (the O*NET portal page that links to several O*NET-related Web sites), and at http://www.doleta.gov/programs/onet (Web site of the Employment and Training Administration at the U.S. Department of Labor). Also see Employment and Training Administration (2009).
2 Of the 923 occupations in the O*NET data collection plan, 853 occupations have been comprehensively updated, and 70 occupations have been partially updated in the May 2023 database version. The 70 partially updated occupations will be comprehensively updated in subsequent database versions.
3 Bright Outlook occupations are those expected to grow rapidly in the next several years or those expected to have large numbers of job openings, or are new and emerging occupations.
4 The goal of the study was to compare the psychometric quality of incumbents’ Skills ratings with that of analysts’ Skills ratings across a large sample of O*NET-SOCs. Although some mean differences between incumbents’ and analysts’ ratings were observed, the results yielded only minimal differences between the two systems of obtaining Skills information.
5 For a description of the pretest, see Supporting Statement Part B, Section 4.1.
6 For a discussion of current and future efforts to improve response rates, see Supporting Statement Part B, Section 4.
7 For additional information about sampling with the Establishment Method, see Supporting Statement Part B, Section 2.1.
8 For additional information about sampling with the OE Method, see Supporting Statement Part B, Section 2.2.
9 See Appendix Exhibit B-2 for the list of questionnaires, number of items and scales, and data sources.
10 For a more detailed discussion of Biemer et al. (2006), see Supporting Statement Part B, Section 5.
11 Survey completion among selected OEs who agree to participate has consistently remained around 90% or higher. At this high level of participation, the need to offer a post survey completion incentive, as for the Establishment Method job incumbents, does not seem warranted.
12 Historically, it has been assumed that of the 100 occupations completed annually, 75 would be completed via the Establishment Method and 25 via the OE Method. For the December 2021–November 2024 period, the number of OE occupations was increased from 25 to 40 per year to help offset the burden and unknown impacts of COVID-19 on establishments (see Section 15 for further discussion). For the March 2024–February 2027 period, the number of OE occupations has been increased further to 60 per year. This change reflects an ongoing effort to increase the overall proportion of OE occupations in the O*NET database to improve data collection efficiency, as described in Section 3.1.
13 For details about the weighting and estimation procedures, see Supporting Statement Part B, Section 1.1.
14 RSE = the standard error of the mean divided by the mean.
15 O*NET suppression criteria are based on substantive expert recommendations (Peterson et al. 1997a), best practices (Willenborg & De Waal, 1996), and other large government surveys (Klein et al., 2002).
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Bogus, Michelle |
| File Modified | 0000-00-00 |
| File Created | 2024-08-02 |