Note to Reviewer - OEWS Estimation Methodology 2021

NOTE TO REVIEWER

FROM: Laurie Salmon, Chief

Division of Occupational Employment and Wage Statistics

U.S Bureau of Labor Statistics

SUBJECT: Implementing New Estimation Methodology for the Occupational Employment and Wage Statistics (OEWS) Program

INTRODUCTION

The Bureau of Labor Statistics (BLS) is implementing a new model-based estimation methodology, called MB3, for the Occupational Employment and Wage Statistics (OEWS) program. The OEWS May 2021 official estimates, released in the spring of 2022, will be the first to use the new methodology.

BLS has published research estimates using the MB3 methodology for several years and has informed OEWS data users and the public of this change. The details of that notice and outreach are included below.

OVERVIEW

OEWS estimates are based on six panels of survey data collected over a 3-year cycle. The final in-scope sample size when the six panels are combined is approximately 1.1 million establishments. A drawback of the 3-year sample design is the data users’ inabilit­­y to efficiently compare estimates from year to year. Such an ability is critical to developing an understanding of the changing demand for occupations in the labor market. To improve data users’ ability to make year-to-year comparisons, BLS initiated a long-term research project that resulted in a recommendation for a new sample design and estimation method. A comprehensive simulation study showed that the new sample design and estimation method substantially improved the accuracy and reliability of the estimates as well as the ability of data users to make efficient year-to-year comparisons. Although a new sample design has not been implemented, BLS decided to test whether the new estimation method could yield improvements over the current estimation method without changing the sample design. Testing indicated that the accuracy and reliability of the estimates improved greatly over the current approach, so BLS published several years of data using the new estimation method as a research series (see below for more details and links to data). Because of the improvement in accuracy and reliability under the new methodology, as well as its compatibility with future goals of time series incorporation, BLS plans to use the new methodology for the official OEWS estimates, beginning with the spring 2022 release for the May 2021 estimates. The format and appearance of the data provided to users will not change and all current products will continue under the improved methodology.

OUTREACH AND PLANNING

In anticipation of this methodology change and improvement, BLS has conducted outreach to the public and data users through the following mechanisms:

• Public announcement. With the March 2021 publication of the May 2020 reference period, BLS announced the coming change on the OEWS website main page. The statement reads, “With the May 2021 estimates, to be released in spring 2022, the OEWS program will use a new estimation methodology. The new model-based estimation methodology, called MB3, has advantages over the existing methodology, as described in the Monthly Labor Review article "Model-Based Estimates for the Occupational Employment Statistics program." Estimates for the years 2015-2018 were re-calculated using the new estimation methodology and are available as research estimates. Technical information is available in the Survey Methods and Reliability Statement for the MB3 Research Estimates of OEWS. More information will be posted on this page when available.”

  • Occupational Employment and Wage Statistics Homepage

  • Screenshot:

• Monthly Labor Review (MLR) Article. In August 2019, an MLR article describing the advantages of the new methodology was published. As described in the MLR, the quality of the data provided to the public, as measured by variances will improve. In addition, more data may be published as a result of improved quality. And state partners who used OEWS data for non-sampled areas will be able to provide better estimates to their customers.

  • Model-based estimates for the Occupational Employment Statistics program

• Current OMB Clearance for the OEWS Information Collection. The development of the MB3 estimation methodology and a link to the MLR article were included in Supporting Statement Part of the current OMB clearance package under section 2 (b) estimation procedures

• OMB Clearance Package - 2020

• Publishing of research estimates and methodology. BLS has re-calculated and published 4 years of data (2015-2019) using the proposed methods for any users interested in examining a particular subset of data.

  • OEWS MB3 Estimates

  • OEWS MB3 Methods and Reliability Statement

• OEWS Policy Council. The OEWS Policy Council, comprised of state and BLS members have discussed the change over several years. The Policy Council reviewed results of detailed state and BLS analysis of MB3 estimates and supports the methodology change. The council created an MB3 subgroup in 2020, to specifically focus on the transition to MB3 and its impact on state OEWS users. This includes the updating of state-used systems like the LEWIS estimates review system.

• Overviews and Trainings of MB3 Methods for states. Many presentations and trainings have been conducted over the years. Future trainings for states will be conducted as needed and previous trainings and documentation have been posted to the StateWeb intranet site for states to access and review in anticipation of the change.

• BLS Data Users. BLS programs that utilize OEWS data have been notified and provided estimates or microdata from the new estimation system. This includes the Occupational Projections Program, National Compensation Survey (NCS), and Occupational Requirements Survey (ORS) programs. The Occupational Projections Program uses the OEWS estimates as the fundamental input for their projections publications. They have been kept informed and evaluated preliminary estimates using a the MB3 estimation methods. NCS uses OEWS data as an input for the President’s Pay Agent.

• The BLS Technical Advisory Committee (TAC) met November 21^st, 2014 to discuss the topic ‘Occupational Employment Statistics (OES) Redesign: Sampling and Estimation’. BLS’ Office of Employment and Unemployment Statistics, Employment Research and Development staff presented to a panel of participants from BLS, The Bloustein School of Planning and Public Policy at Rutgers University, The Conference Board, The Federal Reserve Bank of Dallas, The Federal Reserve Bank of San Francisco, University of Michigan, and W.E. Upjohn Institute for Employment Research.

2