Supporting_Statement_B_(1220-0170)_2018

Job Openings and Labor Turnover Survey

1220-0170

May 2018

Supporting Statement

Job Opening and Labor Turnover Survey (JOLTS)

B. Collection of Information Employing Statistical Methods

For detailed technical materials on the sample allocation, selection, and estimation methods as well as other related statistical procedures see the BLS Handbook, internal BLS technical reports, and ASA papers listed in the references section of this document. The following is a brief summary of the primary statistical features of JOLTS.

1a. Universe

The Job Openings and Labor Turnover Survey measures the job openings, hires, total separations, quits, layoffs and discharges, and other separations for each month at the national level from a sample of about 16,500 establishments (worksites). The universe for this survey is the Quarterly Contribution Reports (QCR) filed by employers subject to State Unemployment Insurance (UI) laws. The U.S. Bureau of Labor Statistics (BLS) receives these QCR for the Quarterly Census of Employment and Wages (QCEW) program from the 50 states, the District of Columbia, Puerto Rico, and the U.S. Virgin Islands. The QCEW data, which are compiled for each calendar quarter, provide a comprehensive business name and address file with employment, wage, detailed geography (i.e., county), and industry information at the six-digit North American Industry Classification System (NAICS) level. This information is provided for over eight million business establishments of which about 8.1 million are in the scope of this survey. Similar data for Federal Government employees covered by the Unemployment Compensation for Federal Employees program (UCFE) are also included. The final data is stored in a Longitudinal Data Base (LDB), which is then used as a sample frame for sample selection. Another data source used for sampling is the universe of railroad establishments obtained from the Federal Railroad Administration.

1b. Sample

Scope—The JOLTS sample is selected from the populations stated above excluding Puerto Rico and the Virgin Islands. It also excludes from the universe records that are for private household workers (NAICS 814110) and records from Agriculture, Forestry, Fishing and Hunting (NAICS 11) other than logging (113310). Records with average employment of zero in the last twelve months are also excluded from the universe.

Stratification—The JOLTS sample has about 16,500 establishments allocated based on the stratification of four census regions, 20 two-digit industry codes, and six employment size classes, including certainty establishments which have a certain level of employment, or the number of establishments in the universe for a sampling cell is less than or equal to 24. These certainty establishments are assigned a sampling weight of 1.00 and other establishments are assigned the sampling weight of the strata population count divided by the strata sample count. The population and sample counts and their employment levels by industry are shown in Table 1.

In addition to the annual sample, BLS added about 250 establishments in each of the three remaining quarters to represent newly formed businesses. The total sample size, therefore, is about 17,000 establishments. However, with a new sample selection every 1^st quarter, the sample size is reduced to about 16,500 after discarding the out of business units not on the current frame.

Table 1: Distribution of Sample by  Industry, April 2018
Industry	IDNAICS	Popn(N)	Popn(Emp)	Sample(n)	Sample(Empl)
Natural resources and mining	21	36,676	758,116	470	118,257
Construction	23	695,300	8,513,793	686	189,028
Nondurable goods	31	121,041	5,134,536	599	306,870
Durable goods	33	205,431	8,369,859	767	785,818
Wholesale Trade	42	556,788	6,558,704	600	225,291
Retail trade	44	990,609	18,376,772	1,370	420,836
Transportation, warehousing, and utilities	48	236,100	6,468,521	665	753,841
Information	51	142,163	3,235,971	662	527,464
Finance and Insurance	52	453,098	6,483,136	517	320,994
Real estate and rental and leasing	53	354,448	2,504,428	549	119,839
Professional and business services	54	1,490,715	19,281,313	1,281	534,476
Employment services	56	83,948	4,554,440	639	409,769
Educational services	61	105,169	3,239,175	651	748,697
Health care and social assistance	62	901,659	20,458,694	1,412	1,755,395
Arts, entertainment, and recreation	71	124,352	3,085,493	622	443,951
Accommodation and food services	72	650,721	16,014,229	1,289	398,329
Other services	81	550,179	4,900,460	463	88,921
Federal government	91	29,711	2,291,038	650	817,011
State and local government education	92	72,673	10,728,375	922	2,555,593
State and local government non-education	93	159,311	9,738,391	791	1,396,294
Total annual sample including certainty/birth units		7,960,092	160,695,444	15,605	12,916,674
Quarterly sample of newly formed businesses				227

2a. Sample Design

Allocation method—The JOLTS sample design is a probability-based stratified random sample. The basic sampling unit is an establishment or worksite which generally remains in the survey for 24 months for a noncertainty establishment and stays out of the survey for the next three years after completion of the 24 months. Important features of the sample design are the use of stratified random sampling, a Neyman allocation (Cochran, 1977, pp. 259-261), and ratio estimators. The characteristics used to stratify the sample are geographic area by four census regions, 2-digit industry divisions as defined in Table 1, and six establishment employment size classes.

JOLTS characteristics are highly correlated with an establishment’s employment level. Thus for a fixed sample size, stratified sampling results in a greater precision than simple random sampling. Given a fixed sample size, the Neyman allocation provides the maximum precision of an estimate. Some establishments are included in the sample with certainty.

Sample Rotation—The sample is divided into one certainty panel (panel 0) and 24 noncertainty panels. Each month, the oldest panel is rotated out and replaced by a new panel. Each panel is asked to provide data for 24 months. This maintains 24 active noncertainty panels for estimation.

In April 2009, new sampling procedures were implemented. During the annual sample, the previously sampled establishments still used in JOLTS estimation were updated, removing out-of-business establishments and updating industry and employment size class information. Also an age variable was added to all establishments in the sample. All the establishments to be used in the JOLTS estimation during the course of the sampling year were then weighted to the current sampling frame, so that they may represent the most current data. During that same sampling year, a quarterly birth sample was also implemented. The purpose of this birth sample is to enroll younger establishments into the JOLTS sample as soon as possible.

2b. Estimation Procedure

The survey utilizes a ratio estimator to improve the precision of the sample estimates. This estimator improves the precision of the sample estimates by utilizing the correlation between the employment data and the characteristics to be measured. A Horvitz-Thompson estimator (Lohr, 1999, Chapter 6.) with a ratio adjustment is used to produce estimates of surveyed characteristics at several levels of geographic and industrial detail. These estimates include the following:

• Totals

• Rates

• Estimates of monthly change

The generalized formula for totals for all survey characteristics (job openings, hires, etc.) for time period t is as follows for ready reference:

where,

is the characteristic of interest for the i^th unit at time t.

is the estimate of a characteristic at time t.

W_t,i is the sample weight for the i^th unit at time t.

NRAF_t,cell is the cell (Region/2-digit NAICS/SZC) non-response adjustment factor defined by

( ) at time t.

Where “respondents” are the all units reporting employment at time t and “eligible” are all sampled units excluding out-of-business units at time t within a cell.

BMF is the (Current Employment Statistics) Benchmark factor at time t. It is computed for each estimation cell as:

Benchmark factor = ( )

where, CES_Emp_t is the employment level at time t obtained from the monthly Current Employment Statistics (CES) Survey, also known as the monthly Payroll Survey. The CES employment serves as a population control for each estimation cell and JOLTS_Emp_t is the sample weighted employment at time t.

The formula for the Job Openings rate is as follows:

where, is the estimated level of job openings at time t.

The generalized formula for all other rates is as follows:

where is the estimate of the characteristic at time t.

Details of JOLTS estimation are available at http://www.bls.gov/osmr/pdf/st000140.pdf

Birth/Death Model—As with any sample survey, the JOLTS sample can be only as current as its sampling frame. The time lag from the birth of an establishment until its appearance on the sampling frame is approximately one year. In addition, many of these new units may fail within the first year. Since these universe units cannot be reflected on the sampling frame immediately, the JOLTS sample cannot capture job openings, hires, and separations from these units during their early existence. BLS has developed a model to estimate birth/death activity using QCEW longitudinal data. The birth/death model also uses historical JOLTS data to estimate the total amount of “churn” (hires and separations) that exists in establishments of various sizes. The model then combines the estimated total churn with the QCEW employment change of younger units (less than 18 months) to estimate the number of hires and separations taking place in these units that cannot be measured through sampling.

The model-based estimate of total separations is distributed to the three components – quits; layoffs and discharges; and other separations – in proportion to their contribution to the sample-based estimate of total separations. Additionally, job openings for the modeled units are estimated by computing the ratio of openings to hires in the collected data and applying that ratio to the modeled hires. The estimates of job openings, hires, and separations produced by the birth/death model are then added to the sample-based estimates produced from the survey to arrive at the estimates for openings, hires, and separations. The derivation of the parameter’s computational procedure is given in Appendix A and Appendix B.

Alignment—JOLTS hires minus separations should be comparable to the CES net employment change. The CES series is considered a highly accurate measure of net employment change owing to its very large sample size and annual benchmarking to universe counts of employment from the QCEW program. However, definitional differences as well as sampling and non-sampling errors between the two surveys historically caused JOLTS to diverge from CES over time. To limit the divergence and to improve the quality of the JOLTS hires and separations series, BLS implemented a monthly alignment method. This monthly alignment method applies the seasonally adjusted CES employment trends to the seasonally adjusted JOLTS implied employment trend (hires minus separations) forcing them to be approximately the same, while preserving the seasonality of the JOLTS data. A brief description is as follows.

First, the two series are seasonally adjusted and the difference between the JOLTS implied employment trend and the CES net employment change is calculated. Next, the JOLTS implied employment trend is adjusted to equal the CES net employment change through a proportional adjustment. This proportional adjustment procedure adjusts the two components (hires, separations) proportionally to their contribution to the total churn (hires plus separations).  For example, if hires are 40 percent of the churn for a given month, they will receive 40 percent of the needed adjustment and separations will receive 60 percent of the needed adjustment. The following example illustrates the adjustment.

Example:  let hires = 40; seps = 60; change in ces emp = -25

1) D = (hires - seps) - change of cesemp = 40 - 60 - (-25) = 5

2) PropAdj_Hires = hires / (hires + seps) * D = 40 / (40 + 60) * 5 = 2

3) PropAdj_Seps = seps / (hires + seps) * D = 60 / (40 + 60) * 5 = 3

4) Hires_sa = Hires - PropAdj_Hires = 40 - 2 = 38

5) Seps_sa = Seps - PropAdj_Seps = 60+3 = 63

Job openings are adjusted based on the adjustment made to hires. This adjustment applies the ratio of job openings to hires to the hires adjustment to arrive at the job openings adjustment. The adjusted job openings, hires, and separations are converted back to not seasonally adjusted data by reversing the application of the original seasonal factors. After the monthly alignment method is used to adjust the not seasonally adjusted level estimates, rate estimates are computed from the adjusted levels. The monthly alignment procedure assures a close match of the JOLTS implied employment trend with the CES trend for not seasonally adjusted data. The adjusted estimates are then again seasonally adjusted (see http://www.bls.gov/osmr/pdf/st090300.pdf).

2c. Reliability

This survey is designed to produce reliable estimates of the characteristics of interest. For the period January 2017 through December 2017, the average relative standard errors for national estimates of job openings; hires; quits; layoffs and discharges; other separations; and total separations rate, respectively, were 1.9, 1.75, 2.12, 4.56, 6.13, and 2.01 percent. (See table 2.) Table 2 details the Average Relative Standard Error of the JOLTS rates in percentages (Coefficients of Variation) for each JOLTS industry and variable. For the most part, the average JOLTS relative standard errors are below a reasonable acceptable RSE (that is, an RSE of 30 percent). The notable exception would be other separations.  The mean Other Separations rate is generally near zero and this proximity of the mean to zero can substantially, on occasion, raise the RSE above a reasonable RSE.

The estimation of sample variances for the JOLTS survey is accomplished through the method of Balanced Half Samples (BHS) similar to CES. This replication technique uses half samples of the original sample and calculates estimates using those subsamples. The replicate weights in both half samples are modified using Fay’s method of perturbation. The sample variance is calculated by measuring the variability of the estimates made from these subsamples. (For a detailed mathematical presentation of this method, see Handbook of Methods, BLS Chapter 2, Bureau of Labor Statistics, 2011 or http://www.bls.gov/opub/hom/homch2.htm under Reliability of Estimates.)

We compute the replicate estimates using the whole sample rather than only half of the sample, as with the original BRR method. For each replicate, sample units are used with the modified weights :

Where

γ = 0.5, a perturbation factor

G_i = ±1, the replicate indicator

h(α) is the element of the Hadamard matrix (row α for a given column)

f is the sampling fraction

w_i is the selection weight

In the above formula, the factor is not part of the Fay’s procedure, it was added to account for sampling from the finite population.

After we obtain the replicate estimates, we compute the variance using the usual formula:

where A is the number of replicates, 114 for JOLTS from a Hadamard matrix of order 116.

Before estimates of these characteristics are released to the public, they are first screened to ensure that they do not violate the Bureau of Labor Statistics’ (BLS) confidentiality pledge. A promise is made by the Bureau to each respondent that BLS will not release its reported data to the public in a manner which would allow others to identify the establishment, firm, or enterprise. Estimates which fail confidentiality screening based on p-percent rule for disclosure (see Federal Committee on Statistical Methodology Working paper 22) are not published.

2d. Revisions

In order to reflect revisions in the CES (Current Employment Statistics) estimates, the CES second closing revision is reflected in the JOLTS second closing estimates, and the final CES revision is incorporated in the JOLTS estimates on a yearly basis when JOLTS rebenchmarks to CES after CES estimates are benchmarked against the QCEW population.

2e. Specialized Procedures

BLS conducted extensive research to: (1) improve sampling procedures to bring in birth units on a timely basis in order to reduce bias; (2) improve the quality of the reported data in order to reduce response error; and (3) improve data collection procedures in order to increase response rates. The BLS achieved a 64 percent overall unweighted response rate. Therefore, the current respondent annual burden is about 21,650 hours with the targeted goal of a 66 percent unweighted response rate. This calculation is derived as:

Annual burden hours = 16,400 X 0.66 X 12 X 10 / 60 = approximately 21,650 hours.

Where, 16,10 is the total number of establishments in the current annual sample and 100 establishments for each of the three remaining quarters for births; 0.66 is the target goal of response rate; 12 months; 10 minutes per schedule; and 60 minutes. NOTE: The actual burden will vary depending on how many birth establishments are sampled during the whole year.

2f. Data Collection Cycles

JOLTS data are collected every month.

Table 2	Average Relative Standard Error in Percentages (Coefficients of Variation) for Rates by Industry January 2017 - December 2017
ID	Industry/Rates	Job Openings	Hires	Quits	Layoffs and Discharges	Other Separations	Total Separations
TOT	Total	1.90	1.75	2.12	4.56	6.13	2.01
PRI	Total Private	2.06	1.86	2.22	4.78	7.24	2.11
21	Mining and Logging	21.57	18.87	14.48	27.87	35.38	14.09
23	Construction	13.83	8.60	11.49	16.24	42.50	8.40
MFG	Manufacturing	5.96	5.77	6.13	13.68	13.80	6.88
DUR	Durable Goods	7.63	8.00	8.47	14.84	16.61	8.24
NDR	Nondurable Goods	9.09	7.99	7.08	24.64	22.80	11.08
TTU	Trade, Transportation, and Utilities	4.74	3.88	4.80	10.13	13.82	4.60
42	Wholesale Trade	8.76	9.77	15.11	20.65	30.70	10.24
44	Retail Trade	6.33	4.59	5.44	12.18	15.49	5.71
48	Transportation, Warehousing, and Utilities	9.36	8.64	9.21	20.00	23.27	9.49
51	Information	8.17	13.48	12.06	22.24	26.01	11.33
FIR	Financial Activities	7.03	8.68	9.51	20.16	26.34	8.96
52	Finance and Insurance	8.03	10.36	10.29	24.47	28.69	9.93
53	Real Estate and Rental and Leasing	14.03	13.08	15.73	27.41	41.09	14.74
54	Professional and Business Services	4.18	3.86	5.40	6.44	14.52	4.14
55	Employment Services	6.11	7.25	9.79	13.27	19.81	7.78
EHS	Education and Health Services	3.77	3.90	3.95	12.23	11.28	4.62
61	Educational Services	7.89	11.66	9.44	24.42	20.99	11.04
62	Health Care and Social Assistance	4.05	4.07	4.26	13.30	12.46	5.00
L&H	Leisure and Hospitality	4.89	3.68	4.21	14.07	20.69	4.60
71	Arts, Entertainment, and Recreation	11.77	10.10	11.38	24.38	32.67	12.71
72	Accommodation and Food Services	5.26	3.88	4.41	15.70	22.22	4.76
81	Other Services	14.43	11.66	14.08	32.21	36.07	14.85
GOV	Government	3.54	4.47	5.17	12.49	6.60	5.77
91	Federal	6.33	5.94	8.07	6.79	7.34	5.17
S&L	State and Local	4.01	4.96	5.56	14.30	7.99	6.51
SLE	State and Local Education	5.39	8.20	7.01	18.11	11.85	8.16
SLN	State and Local, excluding Education	5.30	5.64	7.39	16.90	9.91	8.04

3. Methods to Maximize Response Rates and Nonresponse Adjustment

3a. Maximize Response Rates

To maximize the response rate for this survey, interviewers initially refine addresses ensuring appropriate contact with the employer. Then, employers are mailed a folder containing a JOLTS brochure and data collection form, along with a cover letter explaining the importance of the survey and the need for voluntary cooperation, and pledging confidentiality. An interviewer calls the establishment after the package is sent and attempts to enroll them into the survey. Nonrespondents and establishments that are reluctant to participate are recontacted by an interviewer specially trained in refusal aversion and conversion. The response rates for February 2018 are shown below in Table 3.

3b. Nonresponse Adjustment

As with other surveys, JOLTS experiences a certain level of nonresponse. To adjust for the nonresponses, JOLTS has divided the nonresponse into two groups: (1) unit nonrespondents and failure to enroll; and (2) item nonresponse. Unit nonrespondents are the establishments that do not report the employment and item non-respondents are the establishments that do report employment but do not report one or more data items, for example, job openings or hires.

The unit nonresponse is treated using a Nonresponse Adjustment Factor (NRAF) as explained in the estimation procedure section of this document and item non-response is adjusted using item imputation. Within each sampling cell, NRAFs are calculated every month based on the ratio of the number of viable establishments to the number of usable respondents in that month. The details regarding the NRAF procedure are given in http://www.bls.gov/osmr/pdf/st950130.pdf. The method used for item imputation is Nearest Neighbor Hot Deck. Details of this procedure are available at http://www.bls.gov/osmr/pdf/st000140.pdf . New methods are being developed to improve on the current item non response imputation.

3c. Nonresponse Bias Assessment and Research

As mentioned earlier, JOLTS has developed a birth/death model of hires and separations based on historical QCEW-LDB data. The model allows for establishment-level estimates of hires and separations for all establishments on the QCEW-LDB. Since the QCEW-LDB serves as the sampling frame for JOLTS, it is possible to produce model hires and separations estimates for all establishments sampled by JOLTS. Consequently, it is possible to compare the model estimates for respondents to non-respondents for establishments in the JOLTS sample. The research indicates the JOLTS respondents differ from non-respondents in one important aspect: the rate of out-of-business establishments for responding sample is much lower than for nonresponding sample. That is, it appeared that establishments exiting the labor force were not likely to report JOLTS data as they exited. The JOLTS birth/death model has been added to the estimation process in an attempt to mitigate this bias.

Table 3: Unweighted and Weighted Unit Response Rates by Industry, February 2018
Industry Division	Sampled (n)	Respondents (n)	Out of Business Respondents	Unweighted Response Rate  %	Weighted Response Rate %
1.0 Total	15,767	9,516	1,049	65%	79%
2.0     Total Private	13,385	7,851	980	63%	78%
2.1  Mining and Logging	519	283	63	62%	83%
2.2  Construction	691	419	55	66%	76%
2.3  Manufacturing	1,385	857	79	66%	83%
2.3.1 Durable Goods	778	462	48	63%	82%
2.3.2 Nondurable Goods	607	395	31	69%	84%
2.4 Transportation, Warehousing, and Utilities	2,649	1,531	240	64%	81%
2.4.1 Wholesale Trade	616	364	39	63%	86%
2.4.2 Retail Trade	1,372	757	161	63%	78%
2.4.3 Transportation, Warehousing, and Utilities	661	410	40	66%	81%
2.5 Information	668	310	58	51%	74%
2.6 Financial Activities	1,084	611	63	60%	78%
2.6.1 Finance and Insurance	539	308	34	61%	78%
2.6.2 Real Estate and Rental and Leasing	545	303	29	59%	77%
2.7 Professional and Business Services	1,313	739	93	61%	76%
2.8 Employment Services	641	308	45	52%	68%
2.9  Educational and Health Services	2,062	1,320	108	68%	79%
2.9.1 Educational Services	647	417	25	67%	80%
2.9.2 Health Care and Social Assistance	1,415	903	83	68%	79%
2.10 Leisure and Hospitality	1,904	1,165	146	66%	77%
2.10.1  Arts, Entertainment, and Recreation	623	392	46	68%	82%
2.10.2 Accommodation and Food Services	1,281	773	100	65%	76%
2.11 Other Services	469	308	30	70%	83%
3.0 Government	2,382	1,665	69	72%	81%
3.1 Federal	657	412	22	65%	68%
3.2 State and Local	1,725	1,253	47	75%	83%
3.2.1 State and Local Education	937	661	30	73%	77%
3.2.2 State and Local, excluding Education	788	592	17	77%	86%

4. Tests

The initial survey’s questionnaire was developed and tested using cognitive design techniques. The questionnaire has been used in production of estimates from December 2000 to the present. A Response Analysis Survey (RAS) was conducted on two major industries—Temporary Help Services and State and Local Government Education—to assess the sources of divergence between the employment change from CES and the implied employment change from JOLTS hires minus separations. In the former industry, businesses have a difficult time reporting hires and separations of temporary help workers. In the latter industry, employers have difficulty reporting hires and separations of student workers. BLS now devotes additional resources to the collection, editing, and review of data for these industries. BLS analysts more closely examine reported data that do not provide a consistent picture over time, and recontact the respondents as necessary. Analysts work with the respondents to adjust their reporting practices as possible. Units that cannot be reconciled but are clearly incorrect on a consistent basis are not used; they are replaced by imputed values using standard techniques.

Periodic tests similar to the RAS are necessary to understand the quality of the reported data and to improve the process in order to reduce sources of error or bias. In the future, BLS may make a nonsubstantive change request of approximately 400 respondent burden hours for future cognitive tests, such as a response analysis survey on the reporting of data items. The questionnaire(s) as well as relevant materials will be provided to OMB at the time of the request.

5. Statistical and Analytical Responsibility

Mr. Edwin Robison, Acting Chief, Statistical Methods Division of the Office of Employment and Unemployment Statistics, is responsible for the statistical aspects of the JOLTS program.

Mr. Robison can be reached on 202-691-6363. As mentioned in the above paragraph, BLS seeks consultation with other outside experts on an as needed basis.

6. References

Bureau of Labor Statistics "Handbook of Methods," Chapter 2, Bureau of Labor Statistics, 2004,

http://www.bls.gov/opub/hom/homch2.htm.

Cochran, William, G., (1977), Sampling Techniques 3^rd Ed., New York, Wiley and Sons, 98, 259-261.

Crankshaw, Mark (April-2008), BLS Comparing the Level of Employment Churn: JOLTS Respondents vs. JOLTS Non-Respondents (attached).

Crankshaw, Mark (July, 2008), Simulating JOLTS Hires and Separations Data Using Historical

QCEW Data, http://www.bls.gov/osmr/pdf/st100140.pdf. (paper attached)

Crankshaw, Mark and Stamas, George (Aug, 1999), "Sample Design in the Job Openings and Labor Turnover Survey," ASA Papers and Proceedings, http://www.bls.gov/osmr/pdf/st000140.pdf.

Federal Committee on Statistical Methodology, Subcommittee on Disclosure Limitation Methodology, "Statistical Policy Working Paper 22," http://www.fcsm.gov/working-papers/SPWP22_rev.pdf.

Goodale, Sarah (July-2008), BLS Internal Document: Adjusting Sampling Weights in The JOLTS Sample (working paper-attached).

Goodale, Sarah and Greene, Darrell (Oct 2009), "Improving the Job Openings and Labor Turnover Survey's Sampling Procedure," ASA Papers and Proceedings, http://www.bls.gov/osmr/pdf/st090040.pdf.

Job Openings and Labor Turnover Statistics survey: Statistical Methods Documentation (2006).

Lohr, Sharon L. (1999), Sampling: Design and Analysis, Brooks/Cole Publishing Company, Chapter 6.

Mueller, Kirk; Stamas, George; and Butani, Shail, "Nonresponse Adjustment In Certainty Strata For An Establishment Survey," http://www.bls.gov/osmr/pdf/st950130.pdf.

Cheng, Edmond; Hudson, Nicole; Kropf, Jurgen; and Mercurio, Jeanine, "The CES/JOLTS Divergence: How to Apply the Monthly Alignment Method to help close the Gap," http://www.bls.gov/osmr/pdf/st090300.pdf.

Appendix A

Deriving levels

The initial approach taken to estimate was to utilize the hires and separations rates for stable, contracting, and expanding units. For each type of unit (stable, contracting, and expanding) the hires and separations data was available at the industry level and size level but not for a combined industry-size level. An approximation was made using industry level data and increasing or decreasing levels for each size class within the industry.

Another approach has been taken. In this approach, a dataset containing JOLTS respondent data from Dec 2000 to April 2008 was created such that all reporters reported two consecutive months of data (a necessary precondition for simulation) and, additionally, all reporters reported both hires and separations. Using this data set, a crude simulation was made such that:

1. For stable units, the hires and separations rates found on page 3 were utilized. The rates were smoothed so that the hires rate equaled the separations rate and the industry-size estimate was made using the initial approach.

2. For expanding units, the hires were set equal to the increase in employment and the separations were set to zero.

3. For contracting units, the separations were set equal to the absolute decrease in employment and the hires were set to zero.

This crude simulation would measure the amount of net churn for a given industry-size cell. Comparing this estimate with the actual reported values would enable one to solve for the underlying churn (and hence level) for all industry-size cells. The difference between the reported value and the net churn is equal to the underlying churn (that is, the hires and separations reported in addition to the net change in employment).

Following is an example to illustrate the technique used to derive levels:

ID: 21 (Mining & Natural resources)

Size: 4 (250-999 employees)

Reported Data

Employment: 1,258,767

Hires: 30,277

Separations: 28,652

Crude Simulated Hires: 19,799 Reported – Simulated: 10,478

Crude Simulated Separations: 16,802 Reported – Simulated: 11,850

= 10,478/1,258,767 = 0.83 %

= 11,850/1,258,767 = 0.94 %

Below are the calculated levels for each industry size:

ID	S	Emp	Orig_Hires	Orig_Seps	C_Impied	C_Implied
21	1	14504	594	648	511	601	0.57%	0.32%
21	2	80094	3081	3094	2424	2252	0.82%	1.05%
21	3	314440	10471	10491	5884	5814	1.46%	1.49%
21	4	1258767	30277	28652	19799	16802	0.83%	0.94%
21	5	1492912	23759	25011	15333	15718	0.56%	0.62%
21	6	29894	366	140	513	449	0.00%	0.00%
23	1	22379	1004	1047	859	809	0.65%	1.06%
23	2	229794	11431	12046	8913	8947	1.10%	1.35%
23	3	569558	32932	32755	20682	21724	2.15%	1.94%
23	4	637288	40631	41288	20984	23587	3.08%	2.78%
23	5	1141391	84884	66046	28781	38916	4.92%	2.38%
23	6	225161	5200	6448	2863	4657	1.04%	0.80%
31	1	27293	493	706	402	604	0.33%	0.37%
31	2	98963	3691	3854	2629	2730	1.07%	1.14%
31	3	971022	25611	28187	16726	19297	0.92%	0.92%
31	4	3160271	67644	82306	31846	45164	1.13%	1.18%
31	5	4188433	99473	108788	40411	53721	1.41%	1.31%
31	6	942869	13428	13310	4075	4996	0.99%	0.88%
33	1	15587	513	576	457	465	0.36%	0.71%
33	2	176236	5770	6437	3925	4154	1.05%	1.30%
33	3	1743483	45242	50697	26280	29533	1.09%	1.21%
33	4	5784226	109121	137136	55017	75608	0.94%	1.06%
33	5	7865330	104141	124366	62973	80668	0.52%	0.56%
33	6	16593811	117190	154901	62815	102656	0.33%	0.31%
42	1	28627	595	704	705	582	0.00%	0.43%
42	2	243266	5724	5763	4829	4138	0.37%	0.67%
42	3	669718	17239	17275	10116	17998	1.06%	0.00%
42	4	928400	22793	25240	9943	13239	1.38%	1.29%
42	5	3191628	49358	58455	21868	29618	0.86%	0.90%
44	1	103130	2655	3048	2828	2016	0.00%	1.00%
44	2	365482	15386	15776	10321	11012	1.39%	1.30%
44	3	1787337	91017	87324	41122	44166	2.79%	2.41%
44	4	1462379	78863	76591	32418	34178	3.18%	2.90%
44	5	2319000	121224	112461	58725	33591	2.70%	3.40%
44	6	373621	39839	48516	20860	20363	5.08%	7.54%
48	1	13379	375	458	325	341	0.37%	0.87%
48	2	320310	4361	4977	5531	5423	0.00%	0.00%
48	3	1182664	21597	20777	17483	11835	0.35%	0.76%
48	4	1218587	39959	40399	18051	19338	1.80%	1.73%
48	5	6260773	130071	110621	51525	52898	1.25%	0.92%
48	6	13594810	202640	209447	82383	99945	0.88%	0.81%
51	1	8418	200	212	175	167	0.30%	0.53%
51	2	87343	2472	2624	1554	1522	1.05%	1.26%
51	3	308407	8081	8687	5336	6222	0.89%	0.80%
51	4	838610	14833	18489	9165	11061	0.68%	0.89%
51	5	1886036	30291	33930	12912	19433	0.92%	0.77%
51	6	1855299	44643	47192	11134	10573	1.81%	1.97%
52	1	17147	321	334	358	337	0.00%	0.00%
52	2	166324	3600	3698	2855	3059	0.45%	0.38%
52	3	641224	14313	14958	8151	8336	0.96%	1.03%
52	4	2358154	49490	45860	26845	19376	0.96%	1.12%
52	5	3832948	64972	73486	28153	37994	0.96%	0.93%
52	6	6305608	97040	106120	28060	21219	1.09%	1.35%
ID	S	Emp	Orig_Hires	Orig_Seps	C_Impied	C_Implied
53	1	13050	428	432	315	336	0.87%	0.74%
53	2	62884	2041	2101	1411	1609	1.00%	0.78%
53	3	164801	7417	7419	4436	4763	1.81%	1.61%
53	4	609318	24652	23280	10965	9799	2.25%	2.21%
53	5	249919	13884	16235	8279	10141	2.24%	2.44%
54	1	46235	1360	1556	1108	1341	0.55%	0.47%
54	2	313919	10425	10683	8440	8186	0.63%	0.80%
54	3	1821071	75467	68074	41635	47733	1.86%	1.12%
54	4	3695469	153032	137626	67723	67909	2.31%	1.89%
54	5	5082319	122746	133124	52634	72164	1.38%	1.20%
54	6	10917197	174524	194316	97378	94517	0.71%	0.91%
56	1	3562	135	146	148	138	0.00%	0.22%
56	2	11907	883	760	547	1054	2.82%	0.00%
56	3	49219	8355	6761	2703	2491	11.48%	8.68%
56	4	71476	9800	8305	3433	4859	8.91%	4.82%
56	5	211376	19185	17350	5540	6550	6.46%	5.11%
56	6	1105696	116331	108415	12693	14129	9.37%	8.53%
61	1	19363	366	323	380	351	0.00%	0.00%
61	2	73520	2055	1745	2044	1812	0.01%	0.00%
61	3	298031	8374	6989	7330	6292	0.35%	0.23%
61	4	1048565	25114	21040	27659	26735	0.00%	0.00%
61	5	3790949	71597	64193	77364	67090	0.00%	0.00%
61	6	9204829	143095	103710	127526	123120	0.17%	0.00%
62	1	43209	1410	1356	1008	934	0.93%	0.98%
62	2	308251	9434	9047	6529	6021	0.94%	0.98%
62	3	2094016	72128	63981	31118	33679	1.96%	1.45%
62	4	4689700	125028	101436	49086	41601	1.62%	1.28%
62	5	23037096	416536	314362	149685	126420	1.16%	0.82%
62	6	29557101	458090	326803	153575	74406	1.03%	0.85%
71	1	8879	399	413	610	587	0.00%	0.00%
71	2	52249	3553	3290	2642	2682	1.74%	1.16%
71	3	204795	16241	14560	14019	13822	1.08%	0.36%
71	4	838029	61092	55543	46479	45513	1.74%	1.20%
71	5	3298756	270103	255808	166759	174070	3.13%	2.48%
71	6	299834	10546	8903	4326	4100	2.07%	1.60%
72	1	46600	1722	1525	1109	3739	1.32%	0.00%
72	2	393101	25409	25245	13450	13955	3.04%	2.87%
72	3	833812	54413	51148	25741	28916	3.44%	2.67%
72	4	919954	45839	43999	22205	25571	2.57%	2.00%
72	5	4091228	156671	152035	73334	77568	2.04%	1.82%
72	6	2349840	51009	45821	18157	18343	1.40%	1.17%
81	1	24837	591	708	501	562	0.36%	0.59%
81	2	124960	3852	4274	3084	3276	0.61%	0.80%
81	3	298374	14960	13703	8678	8376	2.11%	1.79%
81	4	684543	31511	27756	16681	18698	2.17%	1.32%
81	5	1256498	60786	41757	26287	24561	2.75%	1.37%
91	1	3194679	49421	45021	23179	11087	0.82%	1.06%
91	2	10318038	158249	128050	49293	37810	1.06%	0.87%
91	3	3055757	39851	38232	46782	17132	0.00%	0.69%
91	4	2656688	49661	43392	22031	12157	1.04%	1.18%
91	5	13969519	222103	200263	58426	63754	1.17%	0.98%
91	6	45700741	585870	454070	269931	166348	0.69%	0.63%
92	1	208970	2701	2438	5013	4114	0.00%	0.00%
92	2	379681	5221	4214	9235	7343	0.00%	0.00%
92	3	2252458	34030	26720	47121	42958	0.00%	0.00%
92	4	4586257	69058	48199	112835	102262	0.00%	0.00%
ID	S	Emp	Orig_Hires	Orig_Seps	C_Impied	C_Implied		Resid_S%
92	5	18518842	311435	207704	396109	356746	0.00%	0.00%
92	6	103749630	1880372	1508904	1480803	1404892	0.39%	0.10%
93	1	112428	2099	1485	1441	1637	0.59%	0.00%
93	2	494395	8066	7322	5725	5367	0.47%	0.40%
93	3	4712258	58616	50842	35485	35649	0.49%	0.32%
93	4	9238622	147199	132673	85143	90945	0.67%	0.45%
93	5	22185518	338298	287402	161892	144201	0.80%	0.65%
93	6	62082500	748715	591932	358277	298834	0.63%	0.47%

NOTE: Negative values were set to 0.00%

NOTE: For the simulation the levels were rounded to the nearest tenth of a percentage point.

A simulation was performed on the JOLTS data and a comparison was made against the actual reported data. Here are the results:

ID	Type	N	Emp	Avg Emp	H	TS	HR	TSR	CR
21	Rep	14,153	3,190,611	225	68,548	68,036	2.15%	2.13%	4.28%
21	Sim	14,153	3,190,611	225	69,588	66,854	2.18%	2.10%	4.28%
23	Rep	33,114	2,825,571	85	176,082	159,630	6.23%	5.65%	11.88%
23	Sim	33,114	2,825,571	85	175,866	160,117	6.22%	5.67%	11.89%
31	Rep	30,963	9,388,851	303	210,340	237,151	2.24%	2.53%	4.77%
31	Sim	30,963	9,388,851	303	207,857	238,280	2.21%	2.54%	4.75%
33	Rep	52,305	32,178,673	615	381,977	474,083	1.19%	1.47%	2.66%
33	Sim	52,305	32,178,673	615	383,807	467,009	1.19%	1.45%	2.64%
42	Rep	28,141	5,061,639	180	95,709	107,437	1.89%	2.12%	4.01%
42	Sim	28,141	5,061,639	180	96,820	108,890	1.91%	2.15%	4.06%
44	Rep	62,609	6,410,949	102	348,984	343,716	5.44%	5.36%	10.80%
44	Sim	62,609	6,410,949	102	349,473	343,704	5.45%	5.36%	10.81%
48	Rep	21,943	22,590,523	1,030	399,003	386,679	1.77%	1.71%	3.48%
48	Sim	21,943	22,590,523	1,030	405,638	385,305	1.80%	1.71%	3.50%
51	Rep	12,190	4,984,113	409	100,520	111,134	2.02%	2.23%	4.25%
51	Sim	12,190	4,984,113	409	100,344	112,266	2.01%	2.25%	4.27%
52	Rep	22,861	13,321,405	583	229,736	244,456	1.72%	1.84%	3.56%
52	Sim	22,861	13,321,405	583	228,934	254,784	1.72%	1.91%	3.63%
53	Rep	12,557	1,099,972	88	48,422	49,467	4.40%	4.50%	8.90%
53	Sim	12,557	1,099,972	88	47,950	49,355	4.36%	4.49%	8.85%
54	Rep	57,411	21,876,210	381	537,554	545,379	2.46%	2.49%	4.95%
54	Sim	57,411	21,876,210	381	537,036	545,247	2.45%	2.49%	4.95%
ID	Type	N	Emp	Avg Emp	H	TS	HR	TSR	CR
56	Rep	2,764	1,453,236	526	154,689	141,737	10.64%	9.75%	20.40%
56	Sim	2,764	1,453,236	526	154,891	141,882	10.66%	9.76%	20.42%
61	Rep	15,046	14,435,257	959	250,601	198,000	1.74%	1.37%	3.11%
61	Sim	15,046	14,435,257	959	261,740	226,161	1.81%	1.57%	3.38%
62	Rep	64,890	59,729,373	920	1,082,626	816,985	1.81%	1.37%	3.18%
62	Sim	64,890	59,729,373	920	1,079,522	830,742	1.81%	1.39%	3.20%
71	Rep	14,377	4,702,542	327	361,934	338,517	7.70%	7.20%	14.90%
71	Sim	14,377	4,702,542	327	360,231	340,092	7.66%	7.23%	14.89%
72	Rep	43,329	8,634,535	199	335,063	319,773	3.88%	3.70%	7.58%
72	Sim	43,329	8,634,535	199	332,903	327,243	3.86%	3.79%	7.65%
81	Rep	23,447	2,389,212	102	111,700	88,198	4.68%	3.69%	8.37%
81	Sim	23,447	2,389,212	102	112,234	88,836	4.70%	3.72%	8.42%
91	Rep	10,739	78,895,422	7,347	1,105,155	909,028	1.40%	1.15%	2.55%
91	Sim	10,739	78,895,422	7,347	1,099,790	926,418	1.39%	1.17%	2.57%
92	Rep	46,938	129,695,838	2,763	2,302,817	1,798,179	1.78%	1.39%	3.16%
92	Sim	46,938	129,695,838	2,763	2,378,613	2,174,869	1.83%	1.68%	3.51%
93	Rep	53,067	99,925,721	1,883	1,302,993	1,071,656	1.30%	1.07%	2.38%
93	Sim	53,067	99,925,721	1,883	1,312,940	1,049,607	1.31%	1.05%	2.36%
ALL	Rep	622,844	522,789,653	839	9,604,453	8,409,241	1.84%	1.61%	3.45%
ALL	Sim	622,844	522,789,653	839	9,696,177	8,837,661	1.85%	1.69%	3.55%

Appendix B

The Current JOLTS imputation vs. the Simulation

A sample of the JOLTS dataset mentioned in Appendix A was drawn. The sample consisted of approximately 14% of the dataset. The units sampled received two treatments: (1) using the simulation, hires and separations data were produced; and (2) they had hires and separations data imputed using the current JOLTS imputation algorithm.

The current JOLTS imputation algorithm is a hot-deck nearest neighbor technique. The imputation cell (region/industry) is sorted by reported monthly employment. Units in need of imputation borrow from the closest available donor within the cell with respect to employment. Ties in closeness are broken randomly.

In this treatment we can directly compare the actual reported hires and separations directly against the hires and separations for the simulated and imputed data.

Below is a summary of the analysis:

ID	N	Emp	OHR	OSR	OCR	SHR	SSR	SCR	IHR	ISR	ICR
21	2,002	512,189	1.87%	1.84%	3.71%	1.84%	1.67%	3.51%	1.85%	1.74%	3.58%
23	4,353	374,430	5.70%	5.56%	11.26%	6.21%	5.40%	11.61%	5.29%	5.00%	10.29%
31	3,842	1,209,735	2.41%	2.44%	4.85%	2.16%	2.30%	4.46%	2.09%	2.16%	4.25%
33	6,898	4,207,648	1.25%	1.30%	2.55%	1.12%	1.22%	2.34%	1.26%	1.37%	2.64%
42	3,715	799,824	1.92%	1.96%	3.88%	2.11%	1.75%	3.86%	1.91%	1.80%	3.72%
44	8,293	848,729	5.13%	5.00%	10.13%	5.03%	4.79%	9.82%	4.55%	4.50%	9.04%
48	2,843	2,812,648	1.88%	1.84%	3.72%	1.77%	1.72%	3.48%	1.86%	1.87%	3.73%
51	1,595	709,821	2.13%	2.22%	4.35%	2.05%	2.07%	4.13%	1.87%	2.12%	3.99%
52	2,960	1,644,613	1.62%	1.65%	3.28%	1.74%	1.80%	3.54%	1.65%	1.57%	3.23%
53	1,700	152,889	4.26%	4.35%	8.61%	4.19%	4.62%	8.80%	4.15%	3.82%	7.98%
54	7,312	2,353,207	2.34%	2.06%	4.40%	2.31%	2.46%	4.77%	2.23%	2.18%	4.41%
56	421	346,873	10.67%	8.66%	19.33%	9.75%	9.83%	19.58%	6.02%	6.61%	12.63%
61	2,372	2,495,490	1.46%	1.33%	2.79%	1.74%	1.74%	3.47%	1.61%	1.40%	3.01%
62	9,064	9,680,562	1.71%	1.30%	3.00%	1.68%	1.37%	3.05%	1.67%	1.28%	2.96%
71	2,157	772,826	8.32%	6.55%	14.88%	7.81%	6.66%	14.47%	5.15%	5.40%	10.55%
72	5,763	1,328,931	3.82%	3.40%	7.22%	3.82%	3.49%	7.31%	3.47%	3.13%	6.60%
81	3,128	422,095	4.83%	3.52%	8.34%	4.65%	3.52%	8.16%	3.69%	3.12%	6.82%
91	1,445	10,690,793	1.38%	1.37%	2.75%	1.08%	1.22%	2.30%	1.47%	1.34%	2.82%
92	6,722	21,219,082	1.78%	1.47%	3.25%	1.78%	1.49%	3.27%	1.72%	1.31%	3.03%
93	7,532	16,074,949	1.23%	1.11%	2.34%	1.24%	1.06%	2.30%	1.28%	0.99%	2.27%
ALL	84,117	78,657,334	1.81%	1.60%	3.41%	1.76%	1.60%	3.36%	1.74%	1.49%	3.23%

Main Finding on the 1^st randomly selected sample of reported JOLTS data:

The imputed values show less churn (both hires and especially separations) than do the actual and simulated values.

Reference- attachment.

Comparing the Level of Employment Churn:

JOLTS Respondents vs. JOLTS Non-Respondents

Mark Crankshaw

April 2008

Introduction

One assumption in the JOLTS survey is that the non-respondents to the JOLTS survey do not systematically differ from respondents. This assumption has been questioned by some and it has been asserted that the non-respondents to the JOLTS survey are more volatile with respect to monthly employment than are respondents to the survey; that is, employment churning of non-respondents greatly exceeds the employment churning of respondents. This would imply that the estimated rates of JOLTS variables that measure employment churning, namely hires and separations, are systematically biased in the downward direction.

One way to test the hypothesis that JOLTS non-respondents have greater employment churning than respondents is to match the JOLTS sample to the Longitudinal Database (LDB). The LDB contains historical employment data for all JOLTS records. The absolute month-to-month employment of matched units on the LDB can serve as a proxy for employment churning. Those establishments with a higher absolute average employment change on the LDB could be assumed to have greater levels of employment churn than establishments with lower absolute average employment change.

Making the Comparison

To test whether non-respondents have higher levels of average absolute employment change than respondents we can contrast the average absolute employment change on the LDB for all non-respondents to the JOLTS survey against the average absolute employment change on the LDB for all respondents to the JOLTS survey. If the average absolute employment change for non-respondents is statistically higher than the average absolute employment change for respondents, then the assumption of no difference is violated. However, if there is no statistical difference found, then the assertion that non-respondents systematically differ from the respondents is not backed up by LDB data.

All JOLTS records were matched with the LDB (over the period April, 2005 to June 2007). The absolute employment change was calculated for all matched records. The average absolute employment change for non-respondents and respondents was calculated.

Findings

There was no evidence found to support the hypothesis that non-respondents systematically differ from respondents to the JOLTS survey. Overall, the average absolute employment change from month to month for respondents was 23.88, while for non-respondents it was 19.83. The difference between the two was not statistically significant. This finding of no difference was found across all months analyzed as well as across all industries.

The graph below charts the average absolute employment change across the months analyzed:

The graph below charts the average absolute employment change for all industries:

The table below details the comparison by industry:

Industry	JOLTS ID	Absolute Average Emp change Respondents	Absolute Average Emp change Non-Respondents
Natural Resources	21	8.35	7.18
Construction	23	10.06	8.89
Durable MFG	31	10.91	12.00
Non-Durable MFG	33	20.64	17.36
Wholesale Trade	42	5.07	3.82
Retail Trade	44	7.05	6.23
Transportation, Warehousing, Utilities	48	18.06	16.05
Information	51	15.52	9.97
Finance & Insurance	52	8.68	8.12
Real Estate	53	6.73	3.84
Pro Bus Services	54	14.13	10.28
Employment Services	56	53.12	25.77
Ed Services	61	55.68	41.08
Health Care	62	11.58	11.09
Arts, Entertainment, Recreation	71	45.49	40.49
Accommodation, Food Services	72	7.97	8.85
Other Services	81	6.32	6.20
Fed Government	91	17.61	15.89
State & Local Ed	92	136.99	113.54
State & Local Non-Ed	93	22.33	21.16

NOTE: No differences are statistically significant.

Reference-Attachment:

Developing a Birth/Death Model

Mark Crankshaw

BLS Washington

August 21, 2008

Background

Prior research has indicated that the current JOLTS estimation may not adequately capture the level of churning (hires and separations) actually occurring in the economy. This primarily due to the inability of the JOLTS survey to capture hires data from new and young firms and to capture separations from closing firms. Additionally, the divergence between the implied employment changes yielded through JOLTS hires and separations level estimates and the actual employment changes seen in CES estimation indicates that additional churning (primarily separations) is systematically under-reported to the JOLTS respondents. This finding was further confirmed by the recently conducted Response Analysis Survey (RAS) for the two industries with the largest divergence. These industries are Employment Services (ID56) and State and Local Government (ID92). While improvements in the JOLTS sampling methodology may help mitigate these inadequacies, the bulk of the shortcomings may have to be treated with a model.

To correct for the above inadequacies, a birth/death model has been developed that will address two separate shortcomings:

• The model will attempt to estimate for a given month the level of employment for firms entering the labor force (that is, birth employment). The model will also estimate the level of hires and separations for those birth establishments.

• The model will attempt to estimate for a given month the level of separations for firms exiting the labor force (that is, establishment deaths). Note that these establishments do not contribute to the employment level since firms that have exited the labor force have no employment by definition.

To that end, the LDB simulation of JOLTS hires and separations data will be utilized. (See the paper entitled ‘Simulating JOLTS Hires and Separations Data Using the LDB’ for the details of this method.) The simulation yields estimated employment, hires and separations for those establishments who have entered the labor force for a given month as well as the employment, hires and separations of those establishments that can not be adequately sampled (i.e., establishments less than 12 months old). The simulation also yields estimated separations levels for those establishments who have exited the labor force in a given month.

Birth Employment

The first aspect to be modeled is the level of birth employment (i.e., first time reporters as well as those young firms less than 30 months old) for a given industry for a given month. The birth employment level is taken directly from the monthly simulation of JOLTS data on the LDB. Likewise, the hires and separations levels for the cohort of birth units were taken directly from the simulation.

Death Separations

The separations from the deaths on the LDB were drawn directly from the simulation. Only the first month of each quarter will contain deaths.

Forecasting

Since current LDB data is unavailable when JOLTS estimation is produced, it is not possible to simulate JOLTS birth/death employment, hires and separations. Therefore, it would be necessary to forecast JOLTS birth/death employment, hires and separations. One possible method that can be used to forecast this data would be to use an ARIMA prediction using historical JOLTS birth/death employment, hires and separations data. An ARIMA forecast has been conducted on this data and the forecast performed adequately. It is also possible to forecast using the ratio of CES year ago employment to current employment to adjust birth employment, hires, and separations.

Reference-Attachment:

Addressing JOLTS-CES Divergence

Beginning with the release of January 2009 data on March 10, BLS will implement improvements to the methodology used to generate estimates of hires, separations, and job openings from the Job Openings and Labor Turnover Survey (JOLTS). These changes are designed to improve the measurement of hires, separations, and openings and to more closely align the hires and separations estimates with monthly employment change as measured by the BLS establishment survey.

Research comparing the relationship between JOLTS hires and separations to the monthly employment change measured by the Bureau’s Current Employment Statistics (CES) program (the establishment survey) indicate substantial discrepancies in employment trends over time. While JOLTS does not produce estimates of month-to-month change in employment, an implied employment change can be derived from JOLTS data by subtracting the separations estimate from the hires estimate for a given month. When viewed over time, this derived JOLTS measure of employment change does not track well with the CES, the Bureau’s larger and better-known establishment survey. The CES is designed specifically to measure month-to-month employment change, collects data from a much larger sample, and benchmarks annually to universe employment counts, making CES the more reliable source of monthly employment change. Further, comparison of JOLTS hires and separations data to similar data produced in the Bureau’s Current Population Survey (CPS or household survey) also indicates that JOLTS may be understating the levels of hires and separations.

BLS engaged in a multi-year research project to better understand these two issues, to establish their probable causes, and to develop improvements. As a result of this research, BLS plans to implement improvements in the following areas:

1. Revision of the JOLTS sample design to incorporate new business births more quickly, and to remove business deaths from the frame on a more timely basis;

2. Addition of a birth/death model for JOLTS to provide an estimate of hires, separations, and openings for births which are too new to be captured by the sample and for deaths which often do not get reported during monthly sample collection;

3. Modification to data collection, editing, and review procedures in specific industries where research has indicated a prevalence of particular response errors; and

4. Establishment of a monthly alignment procedure that takes the CES employment change estimates into consideration.

Improvements to the JOLTS Sample Design

Currently, the JOLTS sample is constructed from individual panels of sample units drawn on an annual basis. The full sample consists of one certainty panel made up of large units selected with virtual certainty based on their size, and 24 noncertainty panels. Each year a new set of panels is drawn from the Bureau’s Longitudinal Database (LDB), a product of the Quarterly Census of Employment and Wages (QCEW) program. Each month a new noncertainty panel is rolled into collection, and the oldest noncertainty panel is rolled out. The collection life of a sample panel is therefore 24 months. This means that at any given time the JOLTS sample is constructed from panels from three different sampling frames, the most current being slightly over one year old and the oldest being slightly over three years old. Thus the JOLTS sample design reflects established firms that have been in business for a minimum of one year.

To better reflect the impact of younger establishments in the JOLTS sample, BLS is modifying the JOLTS sample design in the following ways. First, when a new set of panels is selected each year, the birth units in the sample (those not in existence on the previous year’s frame) will be initiated for collection first, rather than waiting until their associated panel is initiated. Second, each quarter the newly updated LDB will be reviewed to identify birth establishments and a supplemental sample of these units will be drawn and added to the survey; at the same time, out-of-business units will be dropped from the sample on a quarterly basis. Thus, the JOLTS sample will be refreshed quarterly rather than annually. Third, the entire sample of old plus new panels will be poststratified and re-weighted annually to represent the most recent sampling frame; at present, this is not done for sample drawn from earlier frames. This procedure will make the sample more efficient than at present.

JOLTS Business Birth/Death Model

As with any sample survey, the JOLTS sample can only be as current as its sampling frame. The sampling frame for JOLTS is drawn from the LDB, which is updated quarterly from files submitted to the BLS QCEW program as part of the State Unemployment Insurance system. The built-in time lag from the birth of an establishment until its appearance on the sampling frame is approximately one year. In addition, many of these new units may fail within the first year. Since these universe units cannot be reflected on the sampling frame immediately, the JOLTS sample cannot capture job openings, hires, and separations from these units during their early existence. To develop data for these units that cannot be measured through sampling, BLS has developed a model to estimate the contribution of these units to the current month estimates. The birth/death model estimates birth/death activity for current month by examining the birth/death activity from previous years on the LDB and projecting forward using the ratio of over-the-year CES employment change. The birth/death model also uses historical JOLTS data to estimate the amount of “churn” (hires plus separations) that exists in establishments of various sizes. The model then combines the estimated churn with the projected employment change to estimate the number of hires and separations taking place in these units that cannot be measured through sampling.

The model-based estimate of total separations is distributed to the three components: quits, layoffs, and other separations, in proportion to their contribution to the sample-based estimate of total separations. Additionally, job openings for the modeled units are estimated by computing the ratio of openings to hires in the collected data and applying that ratio to the modeled hires.

The estimates of job openings, hires, and separations produced by the birth/death modeling process will then be added to the sample-based estimates produced from the survey to arrive at the final estimates for hires, separations, and openings.

Because JOLTS estimates did not previously include this step, addition of the birth/death model will raise the levels and rates of the hires, separations, and openings measured by JOLTS, and allow the series to more accurately reflect the current labor market.

Modifications to Data Collection Procedures

As stated earlier, an implied measure of employment change can be derived from the JOLTS data by subtracting separations from hires for a given month. Aggregating these monthly changes in the current series, however, generally produces employment levels that overstate employment change as measured by CES, at the total nonfarm level. Research into this problem has shown that a significant amount of the divergence between the CES employment levels and the derived JOLTS employment levels can be traced to the Employment Services industry and to the State Government Education industry. In the former industry, businesses have a difficult time reporting hires and separations of temporary help workers. In the latter industry, employers have a difficult time reporting hires and separations of student workers. BLS plans to devote additional resources to the collection, editing, and review of data for these industries. BLS analysts will more closely examine reported data that do not provide a consistent picture over time, and will re-contact the respondents as necessary. Analysts will work with the respondents to adjust their reporting practices as possible. Units that cannot be reconciled but are clearly incorrect on a consistent basis will be dropped from the estimation process and imputed for using existing techniques.

Establishment of an Alignment Procedure

Over time, employment change derived from JOLTS hires minus separations should track well with employment change measured through the CES. However, there are some definitional differences between the series that can cause legitimate differences for individual months. The major reasons for these month-to-month divergences are:

1. The reference periods of the two surveys are different. CES measures employment for the pay period including the 12^th of the month, while JOLTS measures hires and separations for the entire month.

2. CES counts those who worked or received pay for the reference pay period, while JOLTS counts those who were hired or separated during the reference month. It is possible for a person to miss being paid for a given pay period without having been separated.

Both of these definitional differences can result in differing seasonal patterns between the two series, and therefore cause JOLTS to diverge from the CES in the short-term. Over time however, the computation of JOLTS hires minus separations should reflect employment changes that are consistent with the trends measured by the CES. The three changes to JOLTS that have been described above are expected to produce JOLTS series’ that are much more consistent with the CES. The residual divergence will be controlled through a monthly alignment procedure that allows JOLTS to vary from CES for the reasons listed above, while ensuring that the long-term trends in JOLTS hires-minus-separations match those of the CES net employment change.

The goal of this process is to use current monthly CES employment trends to align the JOLTS implied employment trend (hires minus separations) to be approximately the same, but without forcing all the seasonal patterns to be the same between the surveys. This method takes advantage of the fact that the CES employment series for the current reference month is available prior to the production of JOLTS estimates for that same reference month.

The method works as follows:

• Each month, the initially computed seasonally adjusted JOLTS hires-minus-separations employment change estimate is adjusted to equal the CES seasonally adjusted net employment change estimate, through a proportional adjustment of the hires and separations estimates. By comparing the JOLTS and CES seasonally adjusted changes, the alignment procedure preserves legitimate differences in the seasonal patterns of underlying JOLTS and CES

• Proportional adjustment means that the two components (hires, separations) are adjusted in proportion to their contribution to the total churn (hires plus separations). For example, if hires is 40% of the churn for a given month, it will receive 40% of the needed adjustment and separations will receive 60% of the needed adjustment.

• In the next step, these adjusted hires and separations estimates are converted back to not seasonally adjusted data by reversing the application of the original seasonal adjustment factors.

• These trend-corrected not seasonally adjusted series are then put through the standard X-12 ARIMA seasonal adjustment process to create the final seasonally adjusted published series. These final seasonally adjusted series will not precisely equal the CES seasonally adjusted net employment change but will be very similar.

Revisions to Historical Series

The monthly JOLTS series begin with estimates for December 2000. All published estimates back to that point will be revised to reflect the addition of the revised birth-death model and the new alignment procedure, as well as selected adjustments to individual survey reports. New historical series for job openings, hires, total separations, quits, layoffs and discharges, and other separations will replace the currently available series. At that time, tables comparing the original and revised series will also be available.

Reference-Attachment:

Proposed JOLTS Sample Weights Adjustment

Sarah Goodale

July 2008

Background:

The Job Opening and Labor Turnover Survey (JOLTS) is a stratified random sample with a sample size of 16,000 establishments. The 16,000 establishments are distributed over 25 panels; in which 1 panel is a certainty panel and the remaining 24 panels are noncertainty panels. Each month one panel enters the sample (rolls in) while another panel leaves the sample (rolls out).

Each year a sample is sample is drawn, with which 12 panels will be used to enter the JOLTS sample. Since there are 24 panels that are in rotation, 12 panels of the sample can come from the new sample while the remaining panels are from previous samples. There is a possibility that there are 3 different samples present in JOLTS at once. When the first month of the new sample rolls into to JOLTS; there is 1 panel of the new sample, 12 panels of the sample taken the previous year, and 11 panels of the sample taken 2 years prior. Since the sample weights for JOLTS is currently determined when the establishments are selected to be a part of JOLTS, there can also be three different frames in which the establishments weight to. Also once an establishment has been rolled into JOLTS; it is only removed when the panel rolls out of JOLTS.

Younger establishments are represented proportionally for the frame on the current yearly sample selected. However, when this sample is added to the older samples to make up the 24 panels of JOLTS, the younger establishments are then disproportionate to the frame. Also the younger establishments are mostly represented in JOLTS by the most current sample and are not distributed among the different panels of JOLTS. The younger establishments may have different characteristics then the older establishments, and therefore should be properly represented on the sample.

Objectives:

1) To weight all establishments in JOLTS to the current frame

2) To weight the younger establishment to the represent the appropriate amount on the current frame for all 24 panels

3) To provide a birth refresh of new establishments to help improve the distribution of younger units

Procedure:

1. Draw the new annual sample

   1. Draw the sample using the current sampling procedure

   2. Keep the full frame file

   3. Keep the full 24 panel sample

2. Update the previous samples

   1. Create a subset containing the previous two samples

   2. Remove any Out-of-Business Establishments

   3. Place the establishments in the proper stratum

      1. Merge the previous sample subset with the current full frame, keeping the stratum definition of the current frame dataset

3. Assign the age variable

   1. Assign the age variable to each of the datasets

      1. Age = 0 : establishments that come into existence on the JOLTS frame for the first time or since the last frame to the current frame

      2. Age = 1: establishments that have been on the JOLTS frame for a year

      3. Age = 2: establishments that have been on the JOLTS frame for at least two years

   2. Assign a post stratification variable to the samples and the frame

      1. Age = 0 or Age = 1 post stratification is age/industry/size

      2. Age = 2 post stratification is age/region/industry/size

4. Assign the panel to the new 12 panel sample (old samples only have the weights appended)

   1. Separate the new sample (24 panels drawn earlier) remove the certainty units from the sample and find the count of establishments per stratum

      1. Divide the count of establishments by 24 call this amt

   2. Separate the new sample into the groups age = 0 and 1 and age = 2

      1. If age = 0 or 1 then keep only the first 12 panels

   3. All Age = 0 go into panel 1

      1. Assign a new schedule number to the establishments

   4. All Age = 1 go into panel 2

      1. Assign a new schedule number to the establishments

   5. All Age = 2 go into panel 3 – panel 12

      1. Create amount = 10*amt ( amt is the number per stratum in each panel)

      2. Sort the data in age = 2 into the post stratification variable in order of there original schedule number

      3. Assign a sequence number to these elements in the post stratification variable.

         1. Keep only those elements whose sequence number is less than or equal to the amount

      4. Assign the elements to new panel numbers

      5. Join the sample with the certainty units, and age = 0 and 1.

      6. Assign a new schedule number to elements in the sample

5. Calculate the new weight

   1. Join the new sample with the previous sample

   2. Find the counts of the post stratification variable for the frame and the sample (panel 3 – 26 of the 36 panels of the 3 samples)

   3. Using the post stratification calculate the new weights

6. Birth refresh: This will be done in between the yearly samples

   1. Pull all units from the quarter of interest (from the LDB)

   2. Assign the age variable, keeping only the units that are Age = 0

   3. Remove any OOB and OOS units

   4. Assign the post stratification units to the units

   5. Find the counts for the post stratification variable

   6. Find the amount to sample of birth per post strata by

7. Distribute the births in the 3 panels

   1. Q2 birth panel 4 – panel 6

   2. Q3 birth panel 7 – panel 9

   3. Q4 birth panel 10 – panel 12

1. Create the new full sample file

12