Supporting Statement B (1220-0170)

Supporting Statement

Job Opening and Labor Turnover Survey (JOLTS)

B. Collection of Information Employing Statistical Methods

For detailed technical materials on the sample allocation, selection, and estimation methods as well as other related statistical procedures see BLS Handbook , internal BLS technical reports, and ASA papers listed in the references section. The following is a brief summary of the primary statistical features of JOLTS.

1a. Universe

The Job Openings and Labor Turnover Survey measures the job openings, hires, total separations, quits, lay-offs and discharges, and other separations rates for each month at the national level from a sample of about 16,100 establishments (worksites). The universe for this survey consists of the Quarterly Contribution Reports (QCR) filed by employers subject to State Unemployment Insurance (UI) laws. The U.S. Bureau of Labor Statistics (BLS) receives these QCR for the Quarterly Census of Employment and Wages (QCEW) Program from the 50 States, the District of Columbia, Puerto Rico, and the U.S. Virgin Islands. The QCEW data, which are compiled for each calendar quarter, provide a comprehensive business name and address file with employment, wage, detailed geography (i.e., county), and industry information at the six-digit North American Industry Classification System (NAICS) level. This information is provided for over eight million business establishments of which about 7.3 million are in the scope of this survey. Similar data for Federal Government employees covered by the Unemployment Compensation for Federal Employees program (UCFE) are also included. The final data is stored in a Longitudinal Data Base (LDB), which is then used as a sampling frame for sample selection. Other data used for sampling is the universe of railroad establishments obtained from the Federal Railroad Administration.

1b. Sample

Scope—The JOLTS sample is selected from the populations stated above excluding Puerto Rico and the Virgin Islands. It also excludes from the universe records for private household workers (NAICS 814110) and records from Agriculture, Forestry, Fishing and Hunting (NAICS 11) other than logging (113310). Records with average employment of zero in the last twelve months are also excluded from the universe.

Stratification—The JOLTS sample has about 16,100 establishments allocated based on the stratification of four census regions, 20 two-digit industry codes, and six employment size classes, including certainty establishments which have a certain level of employment, or the number of establishments in the universe for a sampling cell is less than or equal to twenty four. These certainty establishments are assigned a sampling weight of 1.00 and other establishments are assigned the sampling weight of the strata population count divided by the strata sample count. The population and sample counts and their employment levels by industry are shown in Table-1.

In addition to the annual sample, BLS is planning to sample about 100 establishments in each of the three remaining quarters to represent newly formed businesses. The total sample size, therefore, is about 16,400 establishments.

Table-1: Distribution of Sample by Industry (April-2011)
Industry	IdNaics	Popn(N)	Popn(Emp)	Sample(n)	Sample(Empl)
Natural resources and mining	21	36,651	657,160	435	97,959
Construction	23	673,648	5,633,372	770	103,462
Nondurable goods	31	116,074	4,435,037	619	279,177
Durable goods	33	205,373	6,957,636	849	587,643
Wholesale Trade	42	551,841	5,401,308	682	162,813
Retail trade	44	964,871	14,304,205	1,573	238,933
Transportation, warehousing, and utilities	48	209,026	4,663,573	628	477,736
Information	51	128,469	2,712,940	520	263,803
Finance and Insurance	52	434,186	5,429,513	538	250,922
Real estate and rental and leasing	53	317,797	1,907,333	420	55,439
Professional and business services	54	1,286,613	13,684,821	1,422	418,533
Employment services	56	63,319	2,395,319	486	168,079
Educational services	61	84,004	2,399,259	573	567,925
Health care and social assistance	62	752,041	15,817,303	1,546	1,459,347
Arts, entertainment, and recreation	71	107,848	1,871,927	529	252,441
Accommodation and food services	72	576,191	10,863,013	1,303	287,330
Other services	81	510,070	3,665,532	485	64,611
Federal government	91	30,690	2,041,432	585	658,281
State and local government education	92	68,919	9,635,788	1,126	2,624,121
State and local government non-education	93	154,270	9,016,412	979	1,435,268
Total annual sample		7,271,901	123,492,881	16,068	10,453,823
Quarterly sample of newly formed businesses				300

2a. Sample Design

Allocation method—The JOLTS sample design is a probability based stratified random sampling. The basic sampling unit is an establishment or worksite which generally remains in the survey for twenty four months for a non certainty establishment and stays out of the survey for the next three years after completion of the 24 months. Important features of the sample design are the use of stratified random sampling, a Neyman allocation (Cochran, 1977, pp. 259-261), and ratio estimators. The characteristics used to stratify the sample are geographic area by four census regions, 2-digit industry divisions as defined in Table-1, and six establishment employment size classes.

JOLTS characteristics are highly correlated with an establishment’s employment level. Thus for a fixed sample size, stratified sampling results in a greater precision than simple random sampling. Given a fixed sample size, the Neyman allocation provides the maximum precision of an estimate. Some establishments are included in the sample with certainty.

Sample Rotation—The sample is divided into one certainty panel (panel 0) and 24 non-certainty panels. Each month, one of the oldest panels is rotated out and replaced by a new panel. Each panel is asked to provide data for 24 months. This maintains 24 active non-certainty panels for estimation.

In April 2009, new sampling procedures were implemented. During the annual sample, the previously sampled establishments still used in JOLTS estimation were updated, removing out-of-business establishments and updating industry and employment size class information. Also an age variable was added to all establishments in the sample. All the establishments to be used in the JOLTS estimation during the course of the sampling year were then weighted to the current sampling frame, so that they may represent the most current data. During that same sampling year, a quarterly birth sample was also implemented. This is to capture and enroll younger establishments into the JOLTS sample as soon as possible.

2b. Estimation Procedure

The survey utilizes a ratio estimator to improve the precision of the sample estimates. This estimator improves the precision of the sample estimates by utilizing the correlation between the employment data and the characteristics to be measured. A Horvitz-Thompson estimator (Lohr, 1999, Chapter 6) with a ratio adjustment is used to produce estimates of surveyed characteristics at several levels of geographic and industrial detail. These estimates include the following:

• Totals

• Rates

• Estimates of monthly change

The generalized formula for totals for all survey characteristics (job openings, hires, etc.) for time period t is as follows for ready reference:

where,

is the characteristic of interest for the i^th unit at time t.

is the estimate of a characteristic at time t.

W_t,i is the sample weight at time t for i^th unit.

NRAF_t,cell is the cell (Region/2-digit NAICS/SZC) non-response adjustment factor defined by ( ) at time t.

Where respondents are the all units reporting employment at time t and eligible are all sampled units excluding out-of-business units at time t within a cell.

BMF is the (Current Employment Statistics) Benchmark factor at time t. It is computed for each estimation cell as:

Benchmark factor = ( )

where, CES_Emp_t is the employment level at time t obtained from the monthly Current Employment Statistics (CES) Survey, also known as the monthly Payroll Survey. The CES employment serves as a population control for each estimation cell and JOLTS_Emp_t is the sample weighted employment at time t.

The formula for the Job Openings rate is as follows:

where, is the estimated level of job openings at time t.

The generalized formula for all other rates is as follows:

Details of JOLTS estimation are available at http://www.bls.gov/osmr/pdf/st000140.pdf.

Birth/Death Model—As with any sample survey, the JOLTS sample can only be as current as its sampling frame. The time lag from the birth of an establishment until its appearance on the sampling frame is approximately one year. In addition, many of these new units may fail within the first year. Since these universe units cannot be reflected on the sampling frame immediately, the JOLTS sample cannot capture job openings, hires, and separations from these units during their early existence. BLS has developed a model to estimate birth/death activity for current months by examining the birth/death activity from previous years on the QCEW and projecting forward using the ratio of over-the-year CES employment change. The birth/death model also uses historical JOLTS data to estimate the amount of “churn” (hires and separations) that exists in establishments of various sizes. The model then combines the estimated churn with the projected employment change to estimate the number of hires and separations taking place in these units that cannot be measured through sampling.

The model-based estimate of total separations is distributed to the three components – quits; layoffs and discharges; and other separations – in proportion to their contribution to the sample-based estimate of total separations. Additionally, job openings for the modeled units are estimated by computing the ratio of openings to hires in the collected data and applying that ratio to the modeled hires. The estimates of job openings, hires, and separations produced by the birth/death model are then added to the sample-based estimates produced from the survey to arrive at the estimates for openings, hires, and separations.

Alignment—JOLTS hires minus separations should be comparable to the CES net employment change. The CES series is considered a highly accurate measure of net employment change owing to its very large sample size and annual benchmarking to universe counts of employment from the QCEW program. However, definitional differences as well as sampling and non-sampling errors between the two surveys historically caused JOLTS to diverge from CES over time. To limit the divergence and to improve the quality of the JOLTS hires and separations series, BLS implemented a monthly alignment method. This monthly alignment method applies the seasonally adjusted CES employment trends to the seasonally adjusted JOLTS implied employment trend (hires minus separations) forcing them to be approximately the same, while preserving the seasonality of the JOLTS data. A brief description is as follows.

First, the two series are seasonally adjusted and the difference between the JOLTS implied employment trend and the CES net employment change is calculated. Next, the JOLTS implied employment trend is adjusted to equal the CES net employment change through a proportional adjustment. This proportional adjustment procedure adjusts the two components (hires, separations) proportionally to their contribution to the total churn (hires plus separations).  For example, if hires are 40 percent of the churn for a given month, they will receive 40 percent of the needed adjustment and separations will receive 60 percent of the needed adjustment. The following example illustrates the adjustment.

Example:   let hires = 40 seps = 60 change of cesemp = -25

1) D = (hires - seps) - change of cesemp = 40 - 60 - (-25) = 5

2) PropAdj_Hires = hires / (hires + seps) * D = 40 / (40 + 60) * 5 = 2

3) PropAdj_Seps = seps / (hires + seps) * D = 60 / (40 + 60) * 5 = 3

4) Hires_sa = Hires - PropAdj_Hires = 40 - 2 = 38

5) Seps_sa = Seps - PropAdj_Seps = 60+3 = 63

Job openings are adjusted based on the adjustment made to hires. This adjustment applies the ratio of job openings to hires to the hires adjustment to arrive at the job openings adjustment. The adjusted job openings, hires, and separations are converted back to not seasonally adjusted data by reversing the application of the original seasonal factors. After the monthly alignment method is used to adjust the not seasonally adjusted level estimates, rate estimates are computed from the adjusted levels. The monthly alignment procedure assures a close match of the JOLTS implied employment trend with the CES trend for not seasonally adjusted data. The adjusted estimates are then again seasonally adjusted (see http://www.bls.gov/osmr/pdf/st090300.pdf).

2c. Reliability

This survey is designed to produce reliable estimates of the characteristics of interest.

For the period January 2010 through December 2010, the average relative standard errors for national estimates of job openings; hires; quits; layoffs and discharges; other separations; and total separations rate, respectively, were 3.2, 2.5, 2.8, 5.1, 6.8, and 2.8 percent (Table-2).

The estimation of sample variances for the JOLTS survey is accomplished through the method of Balanced Half Samples (BHS) similar to CES. This replication technique uses half samples of the original sample and calculates estimates using those sub samples. The replicate weights in both half-samples are modified using Fay’s method of perturbation. The sample variance is calculated by measuring the variability of the estimates made from these sub samples. (For a detailed mathematical presentation of this method, see Handbook of Methods, BLS Chapter 2, Bureau of Labor Statistics, 2011 or http://www.bls.gov/opub/hom/homch2.htm) under Reliability of Estimates.

We compute the replicate estimates using the whole sample rather than only half of the sample, as with the original BRR method. For each replicate, sample units are used with the modified weights :

(In the above formula, the factor is not part of the Fay’s procedure – this is the way we account for sampling from the finite population).

After we obtain the replicate estimates, we compute the variance using the usual formula:

(1)

Note the squared perturbation factor in the denominator of (1).

Where, A is the number of replicates, in JOLTS case 114 used from a 116 Hadamard matrix

Before estimates of these characteristics are released to the public, they are first screened to ensure that they do not violate the Bureau of Labor Statistics’ (BLS) confidentiality pledge. A promise is made by the Bureau to each respondent that BLS will not release its reported data to the public in a manner which would allow others to identify the establishment, firm, or enterprise. Estimates which fail confidentiality screening based on p-percent rule for disclosure (see Federal Committee on Statistical Methodology Working paper 22) are not published.

2d. Revisions

In order to reflect changes in the CES (Current Employment Statistics), the monthly revision is reflected in the second closing estimates and the final revision is done on a yearly basis as CES estimates are benchmarked against the QCEW population.

2e. Specialized Procedures

BLS conducted extensive research to: 1) improve sampling procedures to bring in birth units on a timely basis in order to reduce bias; 2) improve the quality of the reported data in order to reduce response error; and 3) improve data collection procedures in order to increase response rates. The BLS targeted goal of 66 percent overall un-weighted response rate has been achieved from the earlier level of 56 percent. Therefore, the current respondent yearly burden is about 21,650 hours. This calculation is derived as:

Yearly burden hours= 16,400 X 0.66 X 12 X 10 / 60 = approximately 21,650 hours.

Where, 16,100 is the total number of establishments in the current annual sample and 100 establishments for each of the three remaining quarters for births; 0.66 is the target goal of response rate; 12 months; 10 minutes per schedule; and 60 minutes. NOTE: The actual burden will be a little less as the birth establishments are not in for the whole year.

2f. Data Collection Cycles

JOLTS data are collected every month.

Table-2	Average Relative Standard Error for Rates by Industry January 2010 – December 2010
ID	Industry/Rates	Job Openings	Hires	Quits	Lay-offs	Other Separations	Total Separations
TOT	Total	3.20	2.45	2.76	5.14	6.84	2.84
PRI	Total Private	3.63	2.62	2.90	5.38	8.09	2.99
21	Natural Resources and Mining	17.23	14.89	16.52	20.71	30.48	11.95
23	Construction	36.20	9.84	19.26	15.37	41.71	12.98
MFG	Manufacturing	8.22	8.60	7.92	13.45	17.21	8.45
DUR	Durable Goods	9.57	11.25	10.89	17.71	22.06	11.45
NDR	Nondurable Goods	13.49	11.33	10.29	17.57	19.24	10.76
TTU	Trade Transportation and Utilities	7.75	5.25	4.84	10.63	17.86	5.57
42	Wholesale Trade	13.16	16.66	15.52	20.60	38.17	11.93
44	Retail Trade	10.38	4.50	5.24	13.67	21.57	6.76
48	Transportation Warehousing and Utilities	15.59	15.41	13.35	21.00	25.32	12.39
51	Information	14.97	15.72	13.63	24.56	39.56	14.53
FIR	Financial Activities	12.23	11.86	12.81	21.20	29.08	10.72
52	Finance and Insurance	13.70	14.43	15.22	27.03	32.27	12.21
53	Real Estate and Rental and Leasing	21.56	18.65	21.25	28.84	55.70	18.28
54	Professional Business Services	7.30	6.13	6.25	6.85	13.93	5.02
55	Balance Professional & Business Services	8.55	9.16	9.29	11.04	20.82	7.91
56	Employment Services	14.56	3.73	4.76	4.85	0.00	2.21
EHS	Education and Health Services	4.54	5.62	7.10	11.63	13.34	5.56
61	Private Education Services	11.17	13.13	12.34	23.59	28.52	10.52
62	Health Care and Social Assistance	4.90	6.15	7.78	12.51	14.50	5.89
L&H	Leisure and Hospitality	8.55	4.90	5.61	13.27	19.34	5.25
71	Arts Entertainment and Recreation	17.61	16.51	15.31	24.40	37.83	15.05
72	Accommodation and Food Services	9.24	4.63	5.92	14.61	20.91	5.10
81	Other Services	29.66	11.81	18.96	26.63	43.58	16.76
GOV	Government	4.36	4.91	5.57	12.53	8.21	6.36
91	Federal Government	8.00	7.87	18.44	20.80	12.01	11.34
S&L	State and Local	5.22	5.59	5.72	13.30	9.27	6.88
SLE	State and Local Education	6.09	7.88	6.91	17.21	13.32	7.92
SLN	State and Local Non-Education	7.24	7.70	9.14	16.06	10.44	8.82

3. Methods to Maximize Response Rates and Non Response Adjustment

3a. Maximize Response Rates

To maximize the response rate for this survey, interviewers initially refine addresses ensuring appropriate contact with the employer. Then, employers are mailed a folder containing a JOLTS brochure and data collection form, along with a cover letter explaining the importance of the survey and the need for voluntary cooperation, and pledging confidentiality. An interviewer calls the establishment after the package is sent and attempts to enroll them into the survey. Non-respondents and establishments that are reluctant to participate are re-contacted by an interviewer especially trained in refusal aversion and conversion. The current response rates are shown below in Table-3.

3b. Non Response Adjustment

As with other surveys, JOLTS experiences a certain level of non-response. To adjust for the non-responses, JOLTS has divided the non response into two groups: 1) unit non-respondents and failure to enroll; and 2) item non-response. Unit non-respondents are the establishments who do not report the employment and item non-respondents are the establishments who do report employment and do not report one or more data items, for example, job openings or hires.

The unit non-response is treated using a Non Response Adjustment Factor (NRAF) as explained in the estimation procedure section of this document and item non-response is adjusted using item imputation. Within each sampling cell, NRAFs are calculated every month based on the ratio of the number of viable establishments to the number of usable respondents in that month. The details regarding the NRAF procedure are given in http://www.bls.gov/osmr/pdf/st950130.pdf. The method used for item imputation is Nearest Neighbor Hot Deck. Details of this procedure are available at http://www.bls.gov/osmr/pdf/st000140.pdf .

3c. Non Response Bias Research or Assessment

As mentioned earlier, JOLTS has developed a birth/death model of hires and separations based on historical QCEW-LDB data. The model allows for establishment level estimates of hires and separations for all establishments on the QCEW-LDB. Since the QCEW-LDB serves as the sampling frame for JOLTS, it is possible to produce model hires and separations estimates for all establishments sampled by JOLTS. Consequently, it is possible to compare the model estimates for respondents to non-respondents for establishments in the JOLTS sample. The research indicates the JOLTS respondents differ from non-respondents in one important aspect. The rate of out-of-business establishments for responding sample is much lower than for non-responding sample. That is, it appeared that establishments exiting the labor force were not likely to report JOLTS data as they exited. The JOLTS birth/death model has been added to the estimation process in an attempt to mitigate this bias.

Table-3: Un-weighted and Weighted Unit Response Rates by Industry
Industry Division	Sampled (n)	Respondents (n)	Out of Business Respondents	Un-weighted Response Rate %	Weighted Response Rate %
1.0 Total	16068	10211	548	65.8	75.9
2.0 Total Private	13378	8273	499	64.2	75.8
2.1 Natural Resources & Mining	435	268	12	63.4	76.4
2.2 Construction	770	502	44	69.1	76.5
2.3 Manufacturing	1468	988	59	70.1	81.0
2.3.1 Durable Goods	849	593	33	72.7	82.6
2.3.2 Non Durable Goods	619	395	26	66.6	77.8
2.4 Transportation, Warehousing and Utilities	2883	1662	117	60.1	73.5
2.4.1 Wholesale Trade	682	401	28	61.3	73.0
2.4.2 Retail Trade	1573	848	59	56.0	72.9
2.4.3 Transportation, Warehousing, and Utilities	628	413	30	69.1	77.6
2.5 Information	520	245	23	49.3	63.2
2.6 Financial Activities	958	550	35	59.6	73.2
2.6.1 Finance and Insurance	538	286	17	54.9	71.0
2.6.2 Real Estate and Rental and Leasing	420	264	18	65.7	76.0
2.7 Balance of Professional and Business Services	1422	827	48	60.2	74.7
2.8 Employment Services	486	249	26	54.1	66.1
2.9 Educational and Health Services	2119	1484	52	71.8	81.7
2.9.1 Educational Services	573	384	17	69.1	76.6
2.9.2 Health Care and Social Assistance	1546	1100	35	72.8	82.2
2.10 Leisure and Hospitality	1832	1140	68	64.6	70.5
2.10.1 Arts, Entertainment and Recreation	529	353	17	68.9	77.8
2.10.2 Accommodation and Food Services	1303	787	51	62.9	69.1
2.11 Other Services	485	358	15	76.2	87.3
3.0 Government	2690	1938	49	73.4	79.1
3.1 Federal	585	359	9	62.3	68.0
3.2 State and Local	2105	1579	40	76.5	80.6
3.2.1 State and Local Education	1126	834	18	75.3	78.0
3.2.2 State and Local Non Education	979	745	22	77.8	81.8

4. Tests

The initial survey’s questionnaire was developed and tested using cognitive design techniques. The questionnaire has been used in production of estimates from December 2000 to the present. A Response Analysis Survey (RAS) was conducted on two major industries—Temporary Help Services and State and Local Government Education—to assess the sources of divergence between the employment change from CES and the implied employment change from hires minus separations. In the former industry, businesses have a difficult time reporting hires and separations of temporary help workers.  In the latter industry, employers have difficulty reporting hires and separations of student workers.  BLS now devotes additional resources to the collection, editing, and review of data for these industries.  BLS analysts more closely examine reported data that do not provide a consistent picture over time, and re-contact the respondents as necessary.  Analysts work with the respondents to adjust their reporting practices as possible.  Units that cannot be reconciled but are clearly incorrect on a consistent basis are not used; they are replaced by imputed values using standard techniques.

Periodic tests similar to the recent RAS are necessary to understand the quality of the reported data and to improve the process in order to reduce sources of error or bias. In the future, the JOLTS program may submit a non-substantive change requesting approximately 400 additional respondent burden hours for future cognitive tests, such as a response analysis survey on the reporting of data items.  The questionnaire(s) as well as relevant materials will be provided to OMB at the time of the request.

5. Statistical and Analytical Responsibility

Ms. Shail Butani, Chief, Statistical Methods Division of the Office of Employment and Unemployment Statistics, is responsible for the statistical aspects of the JOLTS program. Ms. Butani can be reached on 202-691-6347. As mentioned in the above paragraph, BLS seeks consultation with other outside experts on an as needed basis.

6. References

Bureau of Labor Statistics "Handbook of Methods", Chapter 2, Bureau of Labor Statistics, 2004,

http://www.bls.gov/opub/hom/homch2.htm .

Cochran, William, G., (1977), Sampling Techniques 3^rd Ed., New York, Wiley and Sons, 98, 259-261.

Crankshaw Mark (April-2008), BLS Comparing the Level of Employment Churn: JOLTS Respondents vs. JOLTS Non-Respondents (attached).

Crankshaw, Mark (July, 2008), Simulating JOLTS Hires and Separations Data Using Historical

QCEW Data, http://www.bls.gov/osmr/pdf/st100140.pdf. (paper attached)

Crankshaw, Mark and Stamas, George (Aug, 1999), "Sample Design in the Job Openings and Labor Turnover Survey," ASA Papers and Proceedings, http://www.bls.gov/osmr/pdf/st000140.pdf

Federal Committee on Statistical Methodology, Subcommittee on Disclosure Limitation Methodology, "Statistical Policy Working Paper 22," http://www.fcsm.gov/working-papers/SPWP22_rev.pdf

Goodale Sarah (July-2008), BLS Internal Document: Adjusting Sampling Weights in The JOLTS Sample (working paper-attached).

Goodale, Sarah and Greene, Darrell (Oct 2009), "Improving the Job Openings and Labor Turnover Survey's Sampling Procedure", ASA Papers and Proceedings, http://www.bls.gov/osmr/pdf/st090040.pdf

Job Openings and Labor Turnover Statistics survey: Statistical Methods Documentation (2006).

Lohr Sharon L. (1999), Sampling: Design and Analysis, Brooks/Cole Publishing Company, Chapter 6.

Mueller, Kirk, Stamas, George, and Butani, Shail, "Nonresponse Adjustment In Certainty Strata For An Establishment Survey," http://www.bls.gov/osmr/pdf/st950130.pdf

Cheng, Edmond, Hudson, Nicole, Kropf, Jurgen, and Mercurio, Jeanine, "The CES/JOLTS Divergence: How to Apply the Monthly Alignment Method to help close the Gap", http://www.bls.gov/osmr/pdf/st090300.pdf

JOLTS Research and Improvement Planning (October 2008), BLS Internal Document

JOLTS Research and Improvement Plan: Technical Appendices (October 2008), BLS Internal Document

Reference- attachment:

Simulating JOLTS Hires and Separations Data

Using the LDB

Mark Crankshaw

BLS Washington

July 2008

Introduction

The JOLTS survey is a 16,000 unit sample of business establishments drawn from a population frame (Longitudinal Data Base) of over 8 million establishments. While the JOLTS sample is allocated and selected with the goal of accurately reflecting the general composition of the population, there is a possibility that the JOLTS sample does not reflect the general composition of the population in certain important regards (namely, with respect to the age). Since it takes a considerable amount of time (8-12 months) to create the frame, allocate and select a sample, and to contact and enroll respondents to the survey, it is likely that the population frame from which the survey is drawn no longer reflects the current population of business establishments, particularly with respect to establishment age. Currently, the JOLTS survey has no way to sample or account for the very young establishments (new businesses) that came into existence during the 8-12 month period of lag needed to enroll establishments into the survey. Additionally the current enrollment procedures lack inclusion of most 1-2 year old units. If these very young establishments systematically differ from relatively older establishments with respect to hires and separations rates, then the JOLTS hires and separations rates may be biased.

Like most surveys, JOLTS experiences a certain level of survey non-response. Therefore, it is possible that the non-respondents to the JOLTS survey differ systematically in some respect to the respondents of the survey and would thus bias JOLTS estimates. A plausible hypothesis is that larger, more established business firms have JOLTS hires and separations data more readily available and therefore report to JOLTS more often than smaller, less established business firms. If smaller, less established firms differ systematically in terms of hires and separations rates than their larger, more established counterparts then JOLTS estimates may be biased. While the current JOLTS nearest-neighbor imputation algorithm should mitigate this effect, it is nonetheless worthwhile to investigate this issue.

The points above lead to a number of important questions regarding the JOLTS survey:

1. To what extent does the JOLTS sample reflect the general composition of the population of business establishments it attempts to estimate with respect to establishment age and size?

2. Does the hires and separations rate of establishments vary with age? How so?

3. To what extent do very young establishments systematically differ from other units?

4. Does the hires and separations rate of establishments vary with size? How so?

5. Do non-respondents differ in a systematic way from respondents? How so?

Methodology

If it were possible to plausibly simulate JOLTS hires and separations rates for all establishments on the population frame for a given time period, it would be possible to address the above questions. The investigation of the divergence between JOLTS hires minus separations and CES employment change is based on a number of logical presuppositions and these logical presuppositions may be utilized to simulate hires and separations data for all establishments on the population frame (henceforth referred to as the LDB).

It is supposed that, for any given firm, that hires minus separations over time should equal the change in employment for that firm. This leads to several useful corollaries:

1. Establishments that experience no change in employment should, on average, have hires rates equal to separations rates.

2. Establishments that are expanding in employment should, on average, have hires rates greater than separations rates.

3. Establishments that are contracting in employment should, on average, have separations rates greater than hires rates.

4. New units (births) should have hires equal at least to first reported employment.

5. Units falling off the frame (deaths) should have separations equal at least to the last reported employment.

To place these corollaries into more precise mathematical terms and using the supposition that a change in employment roughly equals hires minus separations, let M₁ be the employment on the LDB for a given establishment for a given month and let M₂ be the employment on the LDB for a given establishment for the subsequent month:

1. If M₁ = M₂, then H₂=S₂, where H₂ are the hires for the establishment in month 2 and S₂ are the separations for the establishment in month 2.

2. If M₂ > M₁, then H₂= M₂ - M₁ + and S₂ = , where , are an underlying level of churning additional to the expansion in employment.

3. If M₁ > M₂, then H₂= and S₂ = M₁ – M₂ + , where , are an underlying level of churning additional to the contraction in employment.

4. If M₁ = ., M₂ ., that is, the establishment is a birth unit, then H₂= M₂ and S₂ = .

5. If M₂ = ., M₁ ., that is, the establishment is a death unit, then H₂= and S₂ = M₁

Since M₁ and M₂ are known for all establishments on the LDB, simulating hires and separation levels for any establishment could be obtained by generating the appropriate for a given industry/size cell. One way to estimate the appropriate would be to use historical JOLTS reported data. See Appendix A of this document for the method of calculating used in this paper.

Using Historical Reported JOLTS Data to Approximate

An analysis of all JOLTS reported values from Dec 2000 to June 2007 was conducted. Only establishments which reported two consecutive months of data were considered.

Stable Units

As expected, the hires and separations levels of stable employment respondents are approximately equal. The following table details industry level hires and separations rates for units reporting two consecutive months of employment and having M₁ = M₂:

Industry	ID	N	Hires %	TSeps %
Natural Resources & Mining	21	6,156	0.9%	1.2%
Construction	23	15,295	1.6%	1.9%
Nondurable MFG	31	9,790	1.1%	1.2%
Durable MFG	33	14,751	1.3%	1.5%
Wholesale Trade	42	13,846	0.9%	1.0%
Retail Trade	44	29,004	1.9%	1.9%
Transport, Warehousing, Utilities	48	8,212	1.1%	1.2%
Information	51	5,280	1.1%	1.2%
Finance & Insurance	52	10,331	0.9%	0.9%
Real Estate & Rental	53	6,832	1.0%	1.2%
Professional & Business Services	54	26,369	1.2%	1.3%
Employment Services	56	1,232	2.3%	2.2%
Educational Services	61	6,252	0.7%	0.7%
Health Care & Social Assistance	62	22,321	1.6%	1.5%
Arts & Entertainment	71	5,397	1.7%	1.9%
Accommodation & Food	72	15,902	3.0%	2.6%
Other Services	81	13,471	1.2%	1.4%
Federal Government	91	2,497	0.8%	0.8%
State & Local Education	92	14,093	0.4%	0.3%
State & Local Non-Ed	93	14,578	0.7%	0.7%
ALL		241,555	1.41%	1.45%
Size		N	Hires %	Tseps %
1 (1-9 employees)		116,083	1.3%	1.5%
2 (10-49 employees)		78,740	1.5%	1.6%
3 (50-249 employees)		35,981	1.4%	1.4%
4 (250-999 employees)		7,706	1.2%	1.0%
5 (1000-4999 employees)		2,642	1.1%	1.0%
6 (5000+ employees)		403	1.0%	0.8%

Since the hires and separations levels vary by size, the level used in simulation is to be determined at the industry/size level. Since the simulation model is assuming that for a stable employment establishment that hires is equal to separations ( = ), the level will be calculated as: (Hires % + Tseps %)/2. See Appendix A for the final levels and the empirical method for deriving those levels.

Expanding Units

As expected, the hires level of expanding employment respondents is significantly higher than the separations level. The following table details industry level hires and separations rates for units reporting two consecutive months of employment and having M₁ < M₂:

Industry	ID	N	Hires %	TSeps %
Natural Resources & Mining	21	3,828	5.9%	2.3%
Construction	23	8,882	10.7%	3.8%
Nondurable MFG	31	9,367	4.6%	2.0%
Durable MFG	33	17,660	4.1%	2.0%
Wholesale Trade	42	7,076	4.6%	2.0%
Retail Trade	44	17,745	7.5%	3.5%
Transport, Warehousing, Utilities	48	6,662	3.7%	1.6%
Information	51	3,348	3.8%	1.5%
Finance & Insurance	52	6,488	3.4%	1.4%
Real Estate & Rental	53	2,715	8.0%	2.8%
Professional & Business Services	54	16,027	6.9%	2.5%
Employment Services	56	1,068	16.4%	6.8%
Educational Services	61	5,028	3.4%	1.1%
Health Care & Social Assistance	62	24,256	3.8%	2.0%
Arts & Entertainment	71	4,662	10.7%	3.5%
Accommodation & Food	72	13,781	9.4%	4.4%
Other Services	81	5,072	7.5%	2.8%
Federal Government	91	4,322	1.5%	1.0%
State & Local Education	92	19,555	1.5%	0.7%
State & Local Non-Ed	93	21,216	1.8%	0.9%
ALL		198,584	3.68%	1.76%

Size		N	Hires %	Tseps %
1 (1-9 employees)		11,562	15.8%	3.4%
2 (10-49 employees)		39,720	8.5%	3.2%
3 (50-249 employees)		59,234	5.8%	2.6%
4 (250-999 employees)		39,563	3.4%	1.8%
5 (1000-4999 employees)		33,040	2.3%	1.1%
6 (5000+ employees)		15,558	1.8%	1.0%

To estimate the level for expanding units we concentrate on the separations rate since for expanding units S₂ = . See Appendix A for the final levels and the empirical method for deriving those levels.

Contracting Units

As expected, the reported separations level of contracting employment respondents is significantly higher than the reported hires level. The exception to this rule is ID56 (Employment Services) which may be another indication that the reporting of hires and separations data in this industry may be problematic to survey respondents in that industry. The following table details industry level hires and separations rates for units reporting two consecutive months of employment and having M₁ > M₂:

Industry	ID	N	Hires %	TSeps %
Natural Resources & Mining	21	3,647	2.1%	5.4%
Construction	23	8,883	3.5%	10.7%
Nondurable MFG	31	11,054	1.5%	4.1%
Durable MFG	33	19,867	1.3%	3.8%
Wholesale Trade	42	7,342	1.4%	4.5%
Retail Trade	44	17,937	3.2%	6.3%
Transport, Warehousing, Utilities	48	6,802	1.2%	2.5%
Information	51	3,770	1.7%	3.3%
Finance & Insurance	52	6,329	1.6%	2.8%
Real Estate & Rental	53	2,757	3.0%	7.9%
Professional & Business Services	54	15,510	3.2%	5.5%
Employment Services	56	960	13.3%	9.3%
Educational Services	61	4,136	1.4%	3.2%
Health Care & Social Assistance	62	19,871	2.5%	3.5%
Arts & Entertainment	71	4,527	3.4%	10.4%
Accommodation & Food	72	13,660	5.4%	8.3%
Other Services	81	4,699	3.3%	8.3%
Federal Government	91	4,603	1.1%	1.2%
State & Local Education	92	15,040	1.1%	1.2%
State & Local Non-Ed	93	19,820	1.1%	1.5%
ALL		191,214	1.90%	2.94%

Size		N	Hires %	Tseps %
1 (1-9 employees)		15,663	3.6%	25.7%
2 (10-49 employees)		37,814	3.2%	7.9%
3 (50-249 employees)		56,817	2.7%	4.4%
4 (250-999 employees)		39,259	1.9%	2.6%
5 (1000-4999 employees)		28,979	1.5%	1.5%
6 (5000+ employees)		12,682	1.2%	1.2%

To estimate the level for expanding units we concentrate on the hires rate since for contracting units H₂= . See Appendix A for the final levels and the empirical method for deriving those levels.

Preliminary Findings

Using the simulation method detailed above, all establishments on the LDB from November 2005 to June 2007 were given simulated hires and separations levels based on over-the-month change in employment on the LDB. All records on the LDB were assigned an age and size while the simulation produced hires and separations levels for all records on the LDB. (NOTE: It is intended that the simulation will be conducted on LDB going back to at least December 2000).

One aspect of the simulation that is of interest is the impact of establishment age on hires and separations rates. How do hires and separations rates vary with age? To help answer this question all units on the LDB were assigned an age variable based on their first month of reported employment to the LDB. Establishments were classified into six groups: those whose first month of reported employment to the LDB had occurred in the past 12 month prior to the month being simulated were assigned an age of 0; those whose first month of reported employment to the LDB had occurred in the past 13-24 months prior to the month being processed were assigned an age of 1; those whose first month of reported employment to the LDB had occurred in the past 25-36 month prior to the month being processed were assigned an age of 2, and so on up to age 5 (those units which have been reporting to the LDB for 5 or more years). As an example, for simulating June 2007 data, an establishment which had a first month reported employment to the LDB subsequent to June 2006 would have an age of 0.

What was found was that the youngest establishments have hires and separations rates far higher than older establishments, as an establishment ages its relative level of churning decreases, and older establishments are the most numerous and the least dynamic:

Age	MOF	N	AME	AMH	AMTS	HR	TSR
0	1-12	18,078,370	3,854,129	603,299	300,976	15.65%	7.81%
1	13-24	14,322,591	3,708,606	273,759	269,325	7.38%	7.26%
2	25-36	11,980,678	3,739,103	253,826	249,774	6.79%	6.68%
3	37-48	10,279,521	3,778,781	244,429	237,645	6.46%	6.29%
4	49-60	9,174,353	3,968,320	243,993	239,492	6.15%	6.04%
5	61+	108,863,801	113,692,991	4,910,934	4,795,098	4.32%	4.22%
ALL	1+	172,699,314	132,741,930	6,530,240	6,092,310	4.92%	4.59%

MOF: Months on Frame

AME: Average Monthly Employment

AMH: Average Monthly Hires

AMTS: Average Monthly Total Separations

The simulation indicates that the current JOLTS estimates may be underestimating hires and separations rates significantly (the simulation yields a hires and separations rate of 4.92% and 4.59%, respectively, while the JOLTS estimates over the same period of time averaged 3.62% and 3.33%).

One probable reason for the above disparity is the exclusion of young units from the JOLTS sample. The chart below details the distribution with respect to establishment age of the JOLTS sample and the LDB for June 2007:

	POPULATION		WEIGHTED SAMPLE
Age	Emp	Pct	Emp	Pct
0	5,024,815	3.71%	-	0.00%
1	3,670,059	2.71%	481,662	0.36%
2	3,738,119	2.76%	2,940,518	2.18%
3	3,710,561	2.74%	2,242,007	1.66%
4	3,861,397	2.85%	3,835,024	2.84%
5	115,526,040	85.24%	125,390,339	92.96%
ALL	135,530,991	100.00%	134,889,551	100.00%

Another aspect of the simulation that is of interest is the impact of establishment size on hires and separations rates. How do hires and separations rates vary with size? To help answer this question all units on the LDB were assigned a size variable based on their reported employment for the month being simulated. They were classified into six size classes identical to the JOLTS size classification: size 1 (1 to 9 employees); size 2 (10 to 49 employees); size 3 (50 to 249 employees); size 4 (250 to 999 employees); size 5 (1000 to 4999 employees); and size 6 (5000+ employees).

What was found was that the smallest establishments have hires and separations rates far higher than larger establishments, as establishments increase in size their relative level of churning decreases, and larger establishments are the least numerous and the least dynamic:

Size	N	Avg Emp	HR	TSR
1	125,570,289	2.44	6.34%	6.26%
2	36,924,452	18.19	5.68%	5.42%
3	8,874,392	89.94	5.15%	4.58%
4	1,147,934	407.08	4.31%	3.95%
5	169,226	1725.36	3.06%	2.89%
6	13,021	9144.92	2.33%	2.33%
ALL	172,699,314	16.37	4.92%	4.59%

The simulation also allows us to directly compare the hires and separations rates of establishments that were sampled versus those establishments not sampled, and to directly compare the hires and separations rates of establishments who respond to the JOLTS survey versus those who do not respond. The results are summarized in the chart below:

Sampled?	Responded?	N	Avg Emp	HR	TSR
N	N	172,437,423	14.27	5.10%	4.73%
Y	N	100,024	776.44	3.18%	3.36%
Y	Y	161,867	722.61	2.31%	2.38%

NOTE: These are un-weighted estimates.

The establishments that are sampled by the JOLTS survey have lower hires and separations rates than do establishments that are not sampled. The churning rate (hires + separations rates) for those sampled is 5.43% while the churning rate for those not sampled is 9.83%. The fact that smaller establishments sampled by JOLTS have larger sample weights than do larger establishments helps mitigate this disparity, and indeed the weighted estimates of respondents show a churning rate of 6.92%. We would expect that sampled units should have larger average employment than non-sampled units since smaller establishments are sampled with smaller probabilities in the JOLTS sample. However, it does appear that the churning rate of JOLTS respondents even when properly weighted falls far short of the overall churning rate found by the simulation (9.51%).

There may be systematic difference between non-respondents and respondents to the JOLTS survey consistent with the findings of Faberman et al. Smaller establishments are more likely to respond than are medium sized establishments as the chart below shows:

JOLTS Response Rates by Size

Size	N	Response Rate
1	24,808	74.1%
2	42,673	67.6%
3	44,446	58.6%
4	26,999	52.3%
5	19,409	45.0%
6	8,442	60.9%
ALL	166,867	59.4%

Item non-response was investigated to see whether there is a difference in the item response rates by size. It appears that smaller establishments report hires and separations with greater frequency than do larger establishments as the next chart demonstrates:

JOLTS Item Non-Response by Size

Size	N	Non-Response Rate
1	171,931	1.34%
2	195,261	5.98%
3	202,596	12.12%
4	121,498	16.11%
5	15,543	16.97%
6	7,836	19.15%

NOTE: An establishment is an item non-responder if it reports employment and fails to report either hires or separations.

NOTE: Based on JOLTS data from Dec 2000 to March 2008.

Given that smaller establishments have higher churning rates than larger establishments, and that smaller establishments report more frequently (unit and item) to the JOLTS, we would expect that tendency to increase churning rates rather than depressing them. Likewise, if imputation donors are, on average, smaller than imputation recipients, as the JOLTS item non-response data suggests may be the case, then we would expect that tendency to increase churning rates rather than depressing them.

To sum up, although the JOLTS sample does not reflect the general composition of the population of business establishments it attempts to estimate with respect to establishment age and size, of the two aspects investigated, only the establishment age component seems to drive the disparity between the simulated and reported hires and separations rates. The table below details the distribution with respect to establishment size: of the full JOLTS sample and the LDB for June 2007:

	POPULATION		WEIGHTED SAMPLE
Size	Emp	Pct	Emp	Pct
1	15,495,968	11.43%	14,592,078	10.82%
2	34,544,514	25.49%	34,255,654	25.40%
3	41,144,965	30.36%	41,535,190	30.79%
4	23,803,152	17.56%	23,516,068	17.43%
5	14,696,637	10.84%	14,470,791	10.73%
6	5,845,755	4.31%	6,519,769	4.83%
ALL	135,530,991	100.00%	134,889,551	100.00%

The reported hires and separations rates are lower than the simulated hires and separations rates because the JOLTS sample is, on aggregate, comprised of older and therefore less dynamic firms than the population of business establishments it attempts to estimate. Since JOLTS does not capture younger more dynamic firms, and these younger more dynamic firms have higher hires rates than separations rates, it appears that JOLTS has too few hires relative to separations.

Reassessing the Initial Presupposition

Recall that the simulation described in this paper was based on the presupposition that for any given firm, hires minus separations over time should equal the change in employment for that firm. To date, the analysis of the divergence between JOLTS hires minus separations and CES change in employment has been conducted with the assumption that this presupposition should hold for respondents to the JOLTS survey. The magnitude of the divergence may be an indication that respondents to the JOLTS survey do not respond in such a way in which the presupposition that hires minus separations over time should equal the change in employment for that firm. In the simulated data below, the above presupposition is true by default.ⁱ We can compare the actual reported JOLTS data to the simulated JOLTS data to see if the reporters consistently report data for which the presupposition can not hold.

The following table details the comparison of reported value to their simulated counterparts :

Comparison of Reported vs. Simulated Values

ID	Type	N	Emp	Avg Emp	H	TS	HR	TSR	CR
21	Rep	14,153	3,190,611	225	68,548	68,036	2.15%	2.13%	4.28%
21	Sim	14,153	3,190,611	225	69,624	66,796	2.18%	2.09%	4.28%
23	Rep	33,114	2,825,571	85	176,082	159,630	6.23%	5.65%	11.88%
23	Sim	33,114	2,825,571	85	160,149	175,707	5.67%	6.22%	11.89%
31	Rep	30,963	9,388,851	303	210,340	237,151	2.24%	2.53%	4.77%
31	Sim	30,963	9,388,851	303	207,857	238,280	2.21%	2.54%	4.75%
33	Rep	52,305	32,178,673	615	381,977	474,083	1.19%	1.47%	2.66%
33	Sim	52,305	32,178,673	615	384,601	466,218	1.20%	1.45%	2.64%
42	Rep	28,141	5,061,639	180	95,709	107,437	1.89%	2.12%	4.01%
42	Sim	28,141	5,061,639	180	93,798	123,806	1.85%	2.45%	4.30%
44	Rep	62,609	6,410,949	102	348,984	343,716	5.44%	5.36%	10.80%
44	Sim	62,609	6,410,949	102	357,065	336,117	5.57%	5.24%	10.81%
48	Rep	21,943	22,590,523	1,030	399,003	386,679	1.77%	1.71%	3.48%
48	Sim	21,943	22,590,523	1,030	388,231	372,303	1.72%	1.65%	3.37%
51	Rep	12,190	4,984,113	409	100,520	111,134	2.02%	2.23%	4.25%
51	Sim	12,190	4,984,113	409	101,926	110,628	2.05%	2.22%	4.26%
52	Rep	22,861	13,321,405	583	229,736	244,456	1.72%	1.84%	3.56%
52	Sim	22,861	13,321,405	583	236,340	244,239	1.77%	1.83%	3.61%
ID	Type	N	Emp	Avg Emp	H	TS	HR	TSR	CR
53	Rep	12,557	1,099,972	88	48,422	49,467	4.40%	4.50%	8.90%
53	Sim	12,557	1,099,972	88	48,031	49,273	4.37%	4.48%	8.85%
54	Rep	57,411	21,876,210	381	537,554	545,379	2.46%	2.49%	4.95%
54	Sim	57,411	21,876,210	381	529,675	552,607	2.42%	2.53%	4.95%
56	Rep	2,764	1,453,236	526	154,689	141,737	10.64%	9.75%	20.40%
56	Sim	2,764	1,453,236	526	146,173	150,330	10.06%	10.34%	20.40%
61	Rep	15,046	14,435,257	959	250,601	198,000	1.74%	1.37%	3.11%
61	Sim	15,046	14,435,257	959	256,346	231,554	1.78%	1.60%	3.38%
62	Rep	64,890	59,729,373	920	1,082,626	816,985	1.81%	1.37%	3.18%
62	Sim	64,890	59,729,373	920	1,009,102	901,162	1.69%	1.51%	3.20%
71	Rep	14,377	4,702,542	327	361,934	338,517	7.70%	7.20%	14.90%
71	Sim	14,377	4,702,542	327	347,192	353,131	7.38%	7.51%	14.89%
72	Rep	43,329	8,634,535	199	335,063	319,773	3.88%	3.70%	7.58%
72	Sim	43,329	8,634,535	199	321,184	335,280	3.72%	3.88%	7.60%
81	Rep	23,447	2,389,212	102	111,700	88,198	4.68%	3.69%	8.37%
81	Sim	23,447	2,389,212	102	100,414	100,656	4.20%	4.21%	8.42%
91	Rep	10,739	78,895,422	7,347	1,105,155	909,028	1.40%	1.15%	2.55%
91	Sim	10,739	78,895,422	7,347	1,088,651	927,297	1.38%	1.18%	2.56%
92	Rep	46,938	129,695,838	2,763	2,302,817	1,798,179	1.78%	1.39%	3.16%
92	Sim	46,938	129,695,838	2,763	2,310,490	2,024,331	1.78%	1.56%	3.34%
93	Rep	53,067	99,925,721	1,883	1,302,993	1,071,656	1.30%	1.07%	2.38%
93	Sim	53,067	99,925,721	1,883	1,217,128	1,145,798	1.22%	1.15%	2.36%
ALL	Rep	622,844	522,789,653	839	9,604,453	8,409,241	1.84%	1.61%	3.45%
ALL	Sim	622,844	522,789,653	839	9,373,977	8,905,513	1.79%	1.70%	3.50%

In this version of the simulation, the change in monthly employment is forced to equal hires minus separations.

Note: All JOLTS reporters who reported two consecutive months of employment and both hires and separations (Dec 2000 to present)

CR is the churn rate (HR +TSR)

A number of observations can be made:

• The overall churning rate for the simulated data is higher than what was reported; this particularly so in highly seasonal industries such as ID61 (Educational Services) and ID92 (State & Local Education);

• In ID56 (Employment Services) it appears that hires are clearly over-reported while separations are under-reported. This contrasts with ID61 and ID92, in which the difference in churning is only attributable to a lack of separations.

• It appears that respondents may systematically over-report hires and under-report separations

• Although the churn level for the simulation matches in many industries, there are some industries in which the implied reported employment change runs counter to the simulated data. Since the simulated data is, by definition, internally consistent then the data reported for those industries is internally inconsistent.

The presupposition that hires minus separations over time should equal the change in employment for a given firm does not appear to be supported by the reported data. Thus the simulation tends to understate hires rates (by about 0.05%) and tends to significantly overstate separations rates (by about 0.09%). For certain industries, such as Educational Services and State and Local Education, the disparity between churning rate implied with the presupposition and the reported churning rate are easily explained. When school terms begin or end there is a large change in employment (reported to CES and JOLTS and found on the LDB), yet the relationship between employer and employee has not changed (i.e., there is not a corresponding hire or separation).

The above data suggests that respondents to the JOLTS survey systematically under-report separations. It is hypothetically possible that since there may be a time lag between a change in employment (i.e., employee dropped from payroll) and a subsequent separation, it is possible that the separation, when it later occurs, may not be reported to JOLTS.

In addition, this approach to simulation can be used to estimate the hires and separations rates for the aggregation of units that currently JOLTS can not sample, the age 0 (1-12 month old) units. Analysis of monthly birth and death patterns from BED or CPS shed light only upon the initial month that an establishment enters or exits the marketplace. Since with this simulation we can estimate the hires and separations rates of all units on the LDB of a certain age, the simulation can capture how a birth cohort behaves over the course of an entire year rather than for their initial month (or quarter).

The chart below details the hires and separations rate for those units that have been on the LDB for less than 12 months:

Simulated Data (Age 0)⁰

Month	Emp	HR	SR
Nov 2005	3,792,564	11.31%	6.39%
Dec 2005	3,807,421	11.07%	6.69%
Jan 2006	4,083,914	25.05%	12.66%
Feb 2006	3,574,386	11.49%	6.19%
Mar 2006	3,623,216	11.41%	5.64%
Apr 2006	3,980,990	22.29%	9.74%
May 2006	3,626,593	11.87%	6.01%
Jun 2006	3,690,717	13.05%	6.52%
Jul 2006	3,936,569	20.23%	11.02%
Aug 2006	3,629,214	11.69%	6.44%
Sep 2006	3,652,852	12.47%	7.21%
Oct 2006	3,897,720	21.60%	10.46%
Nov 2006	3,555,305	11.56%	6.69%
Dec 2006	3,554,493	10.87%	6.68%
Jan 2007	3,953,639	25.08%	13.18%
Feb 2007	3,485,842	11.91%	6.45%
Mar 2007	3,529,173	11.54%	5.86%
Apr 2007	4,323,978	21.49%	9.96%
May 2007	4,021,732	10.94%	5.60%
Jun 2007	4,051,759	11.20%	6.04%

Such data, properly smoothed to account for the fact that births only appear on the LDB every three months (at the beginning of a quarter), could help serve as a model for the component of the population that JOLTS is currently unable to capture. Such an analysis has the added benefit that it could be conducted as far back as 1990 and thus would incorporate a large range of economic conditions.

Preliminary Conclusion

A simulation of JOLTS hires and separations data was conducted using the LDB. Two major findings were arrived at: 1) the JOLTS sample does not adequately reflect the population with respect to the age of firms, younger firms have much more churning than older firms, hence JOLTS estimated churn rates are too low; 2) JOLTS respondents systematically under-report separations, hence in the aggregate, JOLTS separations rates are too low relative to hires rates. Thus the divergence between JOLTS and CES results from two factors. Not enough hires are reported from the lack of young establishments in the JOLTS sample. Not enough separations are reported to adequately account for employment changes. Neither factor (lack of young establishments, lack of adequate separations) is uniform throughout the industries therefore the divergence found in each industry varies. At the extremes, some industries may have a lot of young establishments and an adequate amount of separations reported, and some may have very few young establishments and an inadequate amount of separations reported. It is therefore possible for different industries to vary with respect to the magnitude of divergence as well as the direction of divergence. A reasonable model for the component of the population that JOLTS is currently unable to capture can also be derived.

Planned Corrective Actions

• Birth Modeling – data such as those presented at the bottom of page 12 could be produced as far back as 1990 for all industries. The employment, hires and separations from the missing cohort of young establishments could be added to monthly estimates prior to benchmarking to CES. Birth Modeling will be updated using the four rolling quarter omega parameters as mentioned in Appendix A for the birth death model. The model estimates will also be updated using the QCEW LDB linkage for proper accounting of birth and deaths for estimates.

Appendix A

Deriving levels

The initial approach taken to estimate was to utilize the hires and separations rates for stable, contracting, and expanding units. For each type of unit (stable, contracting, and expanding) the hires and separations data was available at the industry level and size level but not for a combined industry-size level. An approximation was made using industry level data and increasing or decreasing levels for each size class within the industry.

Another approach has been taken. In this approach, a dataset containing JOLTS respondent data from Dec 2000 to April 2008 was created such that all reporters reported two consecutive months of data (a necessary precondition for simulation) and, additionally, all reporters reported both hires and separations. Using this data set, a crude simulation was made such that:

1. For stable units, the hires and separations rates found on page 3 were utilized. The rates were smoothed so that the hires rate equaled the separations rate and the industry-size estimate was made using the initial approach.

2. For expanding units, the hires were set equal to the increase in employment and the separations were set to zero.

3. For contracting units, the separations were set equal to the absolute decrease in employment and the hires were set to zero.

This crude simulation would measure the amount of net churn for a given industry-size cell. Comparing this estimate with the actual reported values would enable one to solve for the underlying churn (and hence level) for all industry-size cells. The difference between the reported value and the net churn is equal to the underlying churn (that is, the hires and separations reported in addition to the net change in employment).

Following is an example to illustrate the technique used to derive levels:

ID: 21 (Mining & Natural resources)

Size: 4 (250-999 employees)

Reported Data

Employment: 1,258,767

Hires: 30,277

Separations: 28,652

Crude Simulated Hires: 19,799 Reported – Simulated: 10,478

Crude Simulated Separations: 16,802 Reported – Simulated: 11,850

= 10,478/1,258,767 = 0.83 %

= 11,850/1,258,767 = 0.94 %

Below are the calculated levels for each industry size:

ID	S	Emp	Orig_Hires	Orig_Seps	C_Impied	C_Implied
21	1	14504	594	648	511	601	0.57%	0.32%
21	2	80094	3081	3094	2424	2252	0.82%	1.05%
21	3	314440	10471	10491	5884	5814	1.46%	1.49%
21	4	1258767	30277	28652	19799	16802	0.83%	0.94%
21	5	1492912	23759	25011	15333	15718	0.56%	0.62%
21	6	29894	366	140	513	449	0.00%	0.00%
23	1	22379	1004	1047	859	809	0.65%	1.06%
23	2	229794	11431	12046	8913	8947	1.10%	1.35%
23	3	569558	32932	32755	20682	21724	2.15%	1.94%
23	4	637288	40631	41288	20984	23587	3.08%	2.78%
23	5	1141391	84884	66046	28781	38916	4.92%	2.38%
23	6	225161	5200	6448	2863	4657	1.04%	0.80%
31	1	27293	493	706	402	604	0.33%	0.37%
31	2	98963	3691	3854	2629	2730	1.07%	1.14%
31	3	971022	25611	28187	16726	19297	0.92%	0.92%
31	4	3160271	67644	82306	31846	45164	1.13%	1.18%
31	5	4188433	99473	108788	40411	53721	1.41%	1.31%
31	6	942869	13428	13310	4075	4996	0.99%	0.88%
33	1	15587	513	576	457	465	0.36%	0.71%
33	2	176236	5770	6437	3925	4154	1.05%	1.30%
33	3	1743483	45242	50697	26280	29533	1.09%	1.21%
33	4	5784226	109121	137136	55017	75608	0.94%	1.06%
33	5	7865330	104141	124366	62973	80668	0.52%	0.56%
33	6	16593811	117190	154901	62815	102656	0.33%	0.31%
42	1	28627	595	704	705	582	0.00%	0.43%
42	2	243266	5724	5763	4829	4138	0.37%	0.67%
42	3	669718	17239	17275	10116	17998	1.06%	0.00%
42	4	928400	22793	25240	9943	13239	1.38%	1.29%
42	5	3191628	49358	58455	21868	29618	0.86%	0.90%
44	1	103130	2655	3048	2828	2016	0.00%	1.00%
44	2	365482	15386	15776	10321	11012	1.39%	1.30%
44	3	1787337	91017	87324	41122	44166	2.79%	2.41%
44	4	1462379	78863	76591	32418	34178	3.18%	2.90%
44	5	2319000	121224	112461	58725	33591	2.70%	3.40%
44	6	373621	39839	48516	20860	20363	5.08%	7.54%
48	1	13379	375	458	325	341	0.37%	0.87%
48	2	320310	4361	4977	5531	5423	0.00%	0.00%
48	3	1182664	21597	20777	17483	11835	0.35%	0.76%
48	4	1218587	39959	40399	18051	19338	1.80%	1.73%
48	5	6260773	130071	110621	51525	52898	1.25%	0.92%
48	6	13594810	202640	209447	82383	99945	0.88%	0.81%
51	1	8418	200	212	175	167	0.30%	0.53%
51	2	87343	2472	2624	1554	1522	1.05%	1.26%
51	3	308407	8081	8687	5336	6222	0.89%	0.80%
51	4	838610	14833	18489	9165	11061	0.68%	0.89%
51	5	1886036	30291	33930	12912	19433	0.92%	0.77%
51	6	1855299	44643	47192	11134	10573	1.81%	1.97%
52	1	17147	321	334	358	337	0.00%	0.00%
52	2	166324	3600	3698	2855	3059	0.45%	0.38%
52	3	641224	14313	14958	8151	8336	0.96%	1.03%
52	4	2358154	49490	45860	26845	19376	0.96%	1.12%
52	5	3832948	64972	73486	28153	37994	0.96%	0.93%
52	6	6305608	97040	106120	28060	21219	1.09%	1.35%
ID	S	Emp	Orig_Hires	Orig_Seps	C_Impied	C_Implied
53	1	13050	428	432	315	336	0.87%	0.74%
53	2	62884	2041	2101	1411	1609	1.00%	0.78%
53	3	164801	7417	7419	4436	4763	1.81%	1.61%
53	4	609318	24652	23280	10965	9799	2.25%	2.21%
53	5	249919	13884	16235	8279	10141	2.24%	2.44%
54	1	46235	1360	1556	1108	1341	0.55%	0.47%
54	2	313919	10425	10683	8440	8186	0.63%	0.80%
54	3	1821071	75467	68074	41635	47733	1.86%	1.12%
54	4	3695469	153032	137626	67723	67909	2.31%	1.89%
54	5	5082319	122746	133124	52634	72164	1.38%	1.20%
54	6	10917197	174524	194316	97378	94517	0.71%	0.91%
56	1	3562	135	146	148	138	0.00%	0.22%
56	2	11907	883	760	547	1054	2.82%	0.00%
56	3	49219	8355	6761	2703	2491	11.48%	8.68%
56	4	71476	9800	8305	3433	4859	8.91%	4.82%
56	5	211376	19185	17350	5540	6550	6.46%	5.11%
56	6	1105696	116331	108415	12693	14129	9.37%	8.53%
61	1	19363	366	323	380	351	0.00%	0.00%
61	2	73520	2055	1745	2044	1812	0.01%	0.00%
61	3	298031	8374	6989	7330	6292	0.35%	0.23%
61	4	1048565	25114	21040	27659	26735	0.00%	0.00%
61	5	3790949	71597	64193	77364	67090	0.00%	0.00%
61	6	9204829	143095	103710	127526	123120	0.17%	0.00%
62	1	43209	1410	1356	1008	934	0.93%	0.98%
62	2	308251	9434	9047	6529	6021	0.94%	0.98%
62	3	2094016	72128	63981	31118	33679	1.96%	1.45%
62	4	4689700	125028	101436	49086	41601	1.62%	1.28%
62	5	23037096	416536	314362	149685	126420	1.16%	0.82%
62	6	29557101	458090	326803	153575	74406	1.03%	0.85%
71	1	8879	399	413	610	587	0.00%	0.00%
71	2	52249	3553	3290	2642	2682	1.74%	1.16%
71	3	204795	16241	14560	14019	13822	1.08%	0.36%
71	4	838029	61092	55543	46479	45513	1.74%	1.20%
71	5	3298756	270103	255808	166759	174070	3.13%	2.48%
71	6	299834	10546	8903	4326	4100	2.07%	1.60%
72	1	46600	1722	1525	1109	3739	1.32%	0.00%
72	2	393101	25409	25245	13450	13955	3.04%	2.87%
72	3	833812	54413	51148	25741	28916	3.44%	2.67%
72	4	919954	45839	43999	22205	25571	2.57%	2.00%
72	5	4091228	156671	152035	73334	77568	2.04%	1.82%
72	6	2349840	51009	45821	18157	18343	1.40%	1.17%
81	1	24837	591	708	501	562	0.36%	0.59%
81	2	124960	3852	4274	3084	3276	0.61%	0.80%
81	3	298374	14960	13703	8678	8376	2.11%	1.79%
81	4	684543	31511	27756	16681	18698	2.17%	1.32%
81	5	1256498	60786	41757	26287	24561	2.75%	1.37%
91	1	3194679	49421	45021	23179	11087	0.82%	1.06%
91	2	10318038	158249	128050	49293	37810	1.06%	0.87%
91	3	3055757	39851	38232	46782	17132	0.00%	0.69%
91	4	2656688	49661	43392	22031	12157	1.04%	1.18%
91	5	13969519	222103	200263	58426	63754	1.17%	0.98%
91	6	45700741	585870	454070	269931	166348	0.69%	0.63%
92	1	208970	2701	2438	5013	4114	0.00%	0.00%
92	2	379681	5221	4214	9235	7343	0.00%	0.00%
92	3	2252458	34030	26720	47121	42958	0.00%	0.00%
92	4	4586257	69058	48199	112835	102262	0.00%	0.00%
ID	S	Emp	Orig_Hires	Orig_Seps	C_Impied	C_Implied		Resid_S%
92	5	18518842	311435	207704	396109	356746	0.00%	0.00%
92	6	103749630	1880372	1508904	1480803	1404892	0.39%	0.10%
93	1	112428	2099	1485	1441	1637	0.59%	0.00%
93	2	494395	8066	7322	5725	5367	0.47%	0.40%
93	3	4712258	58616	50842	35485	35649	0.49%	0.32%
93	4	9238622	147199	132673	85143	90945	0.67%	0.45%
93	5	22185518	338298	287402	161892	144201	0.80%	0.65%
93	6	62082500	748715	591932	358277	298834	0.63%	0.47%

NOTE: Negative values were set to 0.00%

NOTE: For the simulation the levels were rounded to the nearest tenth of a percentage point.

A simulation was performed on the JOLTS data and a comparison was made against the actual reported data. Here are the results:

ID	Type	N	Emp	Avg Emp	H	TS	HR	TSR	CR
21	Rep	14,153	3,190,611	225	68,548	68,036	2.15%	2.13%	4.28%
21	Sim	14,153	3,190,611	225	69,588	66,854	2.18%	2.10%	4.28%
23	Rep	33,114	2,825,571	85	176,082	159,630	6.23%	5.65%	11.88%
23	Sim	33,114	2,825,571	85	175,866	160,117	6.22%	5.67%	11.89%
31	Rep	30,963	9,388,851	303	210,340	237,151	2.24%	2.53%	4.77%
31	Sim	30,963	9,388,851	303	207,857	238,280	2.21%	2.54%	4.75%
33	Rep	52,305	32,178,673	615	381,977	474,083	1.19%	1.47%	2.66%
33	Sim	52,305	32,178,673	615	383,807	467,009	1.19%	1.45%	2.64%
42	Rep	28,141	5,061,639	180	95,709	107,437	1.89%	2.12%	4.01%
42	Sim	28,141	5,061,639	180	96,820	108,890	1.91%	2.15%	4.06%
44	Rep	62,609	6,410,949	102	348,984	343,716	5.44%	5.36%	10.80%
44	Sim	62,609	6,410,949	102	349,473	343,704	5.45%	5.36%	10.81%
48	Rep	21,943	22,590,523	1,030	399,003	386,679	1.77%	1.71%	3.48%
48	Sim	21,943	22,590,523	1,030	405,638	385,305	1.80%	1.71%	3.50%
51	Rep	12,190	4,984,113	409	100,520	111,134	2.02%	2.23%	4.25%
51	Sim	12,190	4,984,113	409	100,344	112,266	2.01%	2.25%	4.27%
52	Rep	22,861	13,321,405	583	229,736	244,456	1.72%	1.84%	3.56%
52	Sim	22,861	13,321,405	583	228,934	254,784	1.72%	1.91%	3.63%
53	Rep	12,557	1,099,972	88	48,422	49,467	4.40%	4.50%	8.90%
53	Sim	12,557	1,099,972	88	47,950	49,355	4.36%	4.49%	8.85%
54	Rep	57,411	21,876,210	381	537,554	545,379	2.46%	2.49%	4.95%
54	Sim	57,411	21,876,210	381	537,036	545,247	2.45%	2.49%	4.95%
ID	Type	N	Emp	Avg Emp	H	TS	HR	TSR	CR
56	Rep	2,764	1,453,236	526	154,689	141,737	10.64%	9.75%	20.40%
56	Sim	2,764	1,453,236	526	154,891	141,882	10.66%	9.76%	20.42%
61	Rep	15,046	14,435,257	959	250,601	198,000	1.74%	1.37%	3.11%
61	Sim	15,046	14,435,257	959	261,740	226,161	1.81%	1.57%	3.38%
62	Rep	64,890	59,729,373	920	1,082,626	816,985	1.81%	1.37%	3.18%
62	Sim	64,890	59,729,373	920	1,079,522	830,742	1.81%	1.39%	3.20%
71	Rep	14,377	4,702,542	327	361,934	338,517	7.70%	7.20%	14.90%
71	Sim	14,377	4,702,542	327	360,231	340,092	7.66%	7.23%	14.89%
72	Rep	43,329	8,634,535	199	335,063	319,773	3.88%	3.70%	7.58%
72	Sim	43,329	8,634,535	199	332,903	327,243	3.86%	3.79%	7.65%
81	Rep	23,447	2,389,212	102	111,700	88,198	4.68%	3.69%	8.37%
81	Sim	23,447	2,389,212	102	112,234	88,836	4.70%	3.72%	8.42%
91	Rep	10,739	78,895,422	7,347	1,105,155	909,028	1.40%	1.15%	2.55%
91	Sim	10,739	78,895,422	7,347	1,099,790	926,418	1.39%	1.17%	2.57%
92	Rep	46,938	129,695,838	2,763	2,302,817	1,798,179	1.78%	1.39%	3.16%
92	Sim	46,938	129,695,838	2,763	2,378,613	2,174,869	1.83%	1.68%	3.51%
93	Rep	53,067	99,925,721	1,883	1,302,993	1,071,656	1.30%	1.07%	2.38%
93	Sim	53,067	99,925,721	1,883	1,312,940	1,049,607	1.31%	1.05%	2.36%
ALL	Rep	622,844	522,789,653	839	9,604,453	8,409,241	1.84%	1.61%	3.45%
ALL	Sim	622,844	522,789,653	839	9,696,177	8,837,661	1.85%	1.69%	3.55%

Appendix B

The Current JOLTS imputation vs. the Simulation

A sample of the JOLTS dataset mentioned in Appendix A was drawn. The sample consisted of approximately 14% of the dataset. The units sampled received two treatments: 1) using the simulation, hires and separations data were produced 2) they had hires and separations data imputed using the current JOLTS imputation algorithm.

The current JOLTS imputation algorithm is a hot-deck nearest neighbor technique. The imputation cell (region/industry) is sorted by reported monthly employment. Units in need of imputation borrow from the closest available donor within the cell with respect to employment. Ties in closeness are broken randomly.

In this treatment we can directly compare the actual reported hires and separations directly against the hires and separations for the simulated and imputed data.

Below is a summary of the analysis:

ID	N	Emp	OHR	OSR	OCR	SHR	SSR	SCR	IHR	ISR	ICR
21	2,002	512,189	1.87%	1.84%	3.71%	1.84%	1.67%	3.51%	1.85%	1.74%	3.58%
23	4,353	374,430	5.70%	5.56%	11.26%	6.21%	5.40%	11.61%	5.29%	5.00%	10.29%
31	3,842	1,209,735	2.41%	2.44%	4.85%	2.16%	2.30%	4.46%	2.09%	2.16%	4.25%
33	6,898	4,207,648	1.25%	1.30%	2.55%	1.12%	1.22%	2.34%	1.26%	1.37%	2.64%
42	3,715	799,824	1.92%	1.96%	3.88%	2.11%	1.75%	3.86%	1.91%	1.80%	3.72%
44	8,293	848,729	5.13%	5.00%	10.13%	5.03%	4.79%	9.82%	4.55%	4.50%	9.04%
48	2,843	2,812,648	1.88%	1.84%	3.72%	1.77%	1.72%	3.48%	1.86%	1.87%	3.73%
51	1,595	709,821	2.13%	2.22%	4.35%	2.05%	2.07%	4.13%	1.87%	2.12%	3.99%
52	2,960	1,644,613	1.62%	1.65%	3.28%	1.74%	1.80%	3.54%	1.65%	1.57%	3.23%
53	1,700	152,889	4.26%	4.35%	8.61%	4.19%	4.62%	8.80%	4.15%	3.82%	7.98%
54	7,312	2,353,207	2.34%	2.06%	4.40%	2.31%	2.46%	4.77%	2.23%	2.18%	4.41%
56	421	346,873	10.67%	8.66%	19.33%	9.75%	9.83%	19.58%	6.02%	6.61%	12.63%
61	2,372	2,495,490	1.46%	1.33%	2.79%	1.74%	1.74%	3.47%	1.61%	1.40%	3.01%
62	9,064	9,680,562	1.71%	1.30%	3.00%	1.68%	1.37%	3.05%	1.67%	1.28%	2.96%
71	2,157	772,826	8.32%	6.55%	14.88%	7.81%	6.66%	14.47%	5.15%	5.40%	10.55%
72	5,763	1,328,931	3.82%	3.40%	7.22%	3.82%	3.49%	7.31%	3.47%	3.13%	6.60%
81	3,128	422,095	4.83%	3.52%	8.34%	4.65%	3.52%	8.16%	3.69%	3.12%	6.82%
91	1,445	10,690,793	1.38%	1.37%	2.75%	1.08%	1.22%	2.30%	1.47%	1.34%	2.82%
92	6,722	21,219,082	1.78%	1.47%	3.25%	1.78%	1.49%	3.27%	1.72%	1.31%	3.03%
93	7,532	16,074,949	1.23%	1.11%	2.34%	1.24%	1.06%	2.30%	1.28%	0.99%	2.27%
ALL	84,117	78,657,334	1.81%	1.60%	3.41%	1.76%	1.60%	3.36%	1.74%	1.49%	3.23%

Main Finding on the 1^st randomly selected sample of reported JOLTS data:

The imputed values show less churn (both hires and especially separations) than do the actual and simulated values.

Reference- attachment.

Comparing the Level of Employment Churn:

JOLTS Respondents vs. JOLTS Non-Respondents

Mark Crankshaw

April 2008

Introduction

One assumption in the JOLTS survey is that the non-respondents to the JOLTS survey do not systematically differ from respondents. This assumption has been questioned by some and it has been asserted that the non-respondents to the JOLTS survey are more volatile with respect to monthly employment than are respondents to the survey; that is, employment churning of non-respondents greatly exceeds the employment churning of respondents. This would imply that the estimated rates of JOLTS variables that measure employment churning, namely hires and separations, are systematically biased in the downward direction.

One way to test the hypothesis that JOLTS non-respondents have greater employment churning than respondents is to match the JOLTS sample to the Longitudinal Database (LDB). The LDB contains historical employment data for all JOLTS records. The absolute month-to-month employment of matched units on the LDB can serve as a proxy for employment churning. Those establishments with a higher absolute average employment change on the LDB could be assumed to have greater levels of employment churn than establishments with lower absolute average employment change.

Making the Comparison

To test whether non-respondents have higher levels of average absolute employment change than respondents we can contrast the average absolute employment change on the LDB for all non-respondents to the JOLTS survey against the average absolute employment change on the LDB for all respondents to the JOLTS survey. If the average absolute employment change for non-respondents is statistically higher than the average absolute employment change for respondents, then the assumption of no difference is violated. However, if there is no statistical difference found, then the assertion that non-respondents systematically differ from the respondents is not backed up by LDB data.

All JOLTS records were matched with the LDB (over the period April, 2005 to June 2007). The absolute employment change was calculated for all matched records. The average absolute employment change for non-respondents and respondents was calculated.

Findings

There was no evidence found to support the hypothesis that non-respondents systematically differ from respondents to the JOLTS survey. Overall, the average absolute employment change from month to month for respondents was 23.88, while for non-respondents it was 19.83. The difference between the two was not statistically significant. This finding of no difference was found across all months analyzed as well as across all industries.

The graph below charts the average absolute employment change across the months analyzed:

The graph below charts the average absolute employment change for all industries:

The table below details the comparison by industry:

Industry	JOLTS ID	Absolute Average Emp change Respondents	Absolute Average Emp change Non-Respondents
Natural Resources	21	8.35	7.18
Construction	23	10.06	8.89
Durable MFG	31	10.91	12.00
Non-Durable MFG	33	20.64	17.36
Wholesale Trade	42	5.07	3.82
Retail Trade	44	7.05	6.23
Transportation, Warehousing, Utilities	48	18.06	16.05
Information	51	15.52	9.97
Finance & Insurance	52	8.68	8.12
Real Estate	53	6.73	3.84
Pro Bus Services	54	14.13	10.28
Employment Services	56	53.12	25.77
Ed Services	61	55.68	41.08
Health Care	62	11.58	11.09
Arts, Entertainment, Recreation	71	45.49	40.49
Accommodation, Food Services	72	7.97	8.85
Other Services	81	6.32	6.20
Fed Government	91	17.61	15.89
State & Local Ed	92	136.99	113.54
State & Local Non-Ed	93	22.33	21.16

NOTE: No differences are statistically significant.

Reference-Attachment:

Proposed JOLTS Sample Weights Adjustment

Sarah Goodale

July 2008

Background:

The Job Opening and Labor Turnover Survey (JOLTS) is a stratified random sample with a sample size of 16,000 establishments. The 16,000 establishments are distributed over 25 panels; in which 1 panel is a certainty panel and the remaining 24 panels are non-certainty panels. Each month one panel enters the sample (rolls in) while another panel leaves the sample (rolls out).

Each year a sample is sample is drawn, with which 12 panels will be used to enter the JOLTS sample. Since there are 24 panels that are in rotation, 12 panels of the sample can come from the new sample while the remaining panels are from previous samples. There is a possibility that there are 3 different samples present in JOLTS at once. When the first month of the new sample rolls into to JOLTS; there is 1 panel of the new sample, 12 panels of the sample taken the previous year, and 11 panels of the sample taken 2 years prior. Since the sample weights for JOLTS is currently determined when the establishments are selected to be a part of JOLTS, there can also be three different frames in which the establishments weight to. Also once an establishment has been rolled into JOLTS; it is only removed when the panel rolls out of JOLTS.

Younger establishments are represented proportionally for the frame on the current yearly sample selected. However, when this sample is added to the older samples to make up the 24 panels of JOLTS, the younger establishments are then disproportionate to the frame. Also the younger establishments are mostly represented in JOLTS by the most current sample and are not distributed among the different panels of JOLTS. The younger establishments may have different characteristics then the older establishments, and therefore should be properly represented on the sample.

Objectives:

1) To weight all establishments in JOLTS to the current frame

2) To weight the younger establishment to the represent the appropriate amount on the current frame for all 24 panels

3) To provide a birth refresh of new establishments to help improve the distribution of younger units

Procedure:

1. Draw the new annual sample

   1. Draw the sample using the current sampling procedure

   2. Keep the full frame file

   3. Keep the full 24 panel sample

2. Update the previous samples

   1. Create a subset containing the previous two samples

   2. Remove any Out-of-Business Establishments

   3. Place the establishments in the proper stratum

      1. Merge the previous sample subset with the current full frame, keeping the stratum definition of the current frame dataset

3. Assign the age variable

   1. Assign the age variable to each of the datasets

      1. Age = 0 : establishments that come into existence on the JOLTS frame for the first time or since the last frame to the current frame

      2. Age = 1: establishments that have been on the JOLTS frame for a year

      3. Age = 2: establishments that have been on the JOLTS frame for at least two years

   2. Assign a post stratification variable to the samples and the frame

      1. Age = 0 or Age = 1 post stratification is age/industry/size

      2. Age = 2 post stratification is age/region/industry/size

4. Assign the panel to the new 12 panel sample (old samples only have the weights appended)

   1. Separate the new sample (24 panels drawn earlier) remove the certainty units from the sample and find the count of establishments per stratum

      1. Divide the count of establishments by 24 call this amt

   2. Separate the new sample into the groups age = 0 and 1 and age = 2

      1. If age = 0 or 1 then keep only the first 12 panels

   3. All Age = 0 go into panel 1

      1. Assign a new schedule number to the establishments

   4. All Age = 1 go into panel 2

      1. Assign a new schedule number to the establishments

   5. All Age = 2 go into panel 3 – panel 12

      1. Create amount = 10*amt ( amt is the number per stratum in each panel)

      2. Sort the data in age = 2 into the post stratification variable in order of there original schedule number

      3. Assign a sequence number to these elements in the post stratification variable.

         1. Keep only those elements whose sequence number is less than or equal to the amount

      4. Assign the elements to new panel numbers

      5. Join the sample with the certainty units, and age = 0 and 1.

      6. Assign a new schedule number to elements in the sample

5. Calculate the new weight

   1. Join the new sample with the previous sample

   2. Find the counts of the post stratification variable for the frame and the sample (panel 3 – 26 of the 36 panels of the 3 samples)

   3. Using the post stratification calculate the new weights

6. Birth refresh: This will be done in between the yearly samples

   1. Pull all units from the quarter of interest (from the LDB)

   2. Assign the age variable, keeping only the units that are Age = 0

   3. Remove any OOB and OOS units

   4. Assign the post stratification units to the units

   5. Find the counts for the post stratification variable

   6. Find the amount to sample of birth per post strata by

7. Distribute the births in the 3 panels

   1. Q2 birth panel 4 – panel 6

   2. Q3 birth panel 7 – panel 9

   3. Q4 birth panel 10 – panel 12

1. Create the new full sample file

Reference-Attachment:

Developing a Birth/Death Model

Mark Crankshaw

BLS Washington

August 21, 2008

Background

Prior research has indicated that the current JOLTS estimation may not adequately capture the level of churning (hires and separations) actually occurring in the economy. This primarily due to the inability of the JOLTS survey to capture hires data from new and young firms and to capture separations from closing firms. Additionally, the divergence between the implied employment changes yielded through JOLTS hires and separations level estimates and the actual employment changes seen in CES estimation indicates that additional churning (primarily separations) is systematically under-reported to the JOLTS respondents. This finding was further confirmed by the recently conducted Response Analysis Survey (RAS) for the two industries with the largest divergence. These industries are Employment Services (ID56) and State and Local Government (ID92). While improvements in the JOLTS sampling methodology may help mitigate these inadequacies, the bulk of the shortcomings may have to be treated with a model.

To correct for the above inadequacies, a birth/death model has been developed that will address two separate shortcomings:

• The model will attempt to estimate for a given month the level of employment for firms entering the labor force (that is, birth employment). The model will also estimate the level of hires and separations for those birth establishments.

• The model will attempt to estimate for a given month the level of separations for firms exiting the labor force (that is, establishment deaths). Note that these establishments do not contribute to the employment level since firms that have exited the labor force have no employment by definition.

To that end, the LDB simulation of JOLTS hires and separations data will be utilized. (See the paper entitled ‘Simulating JOLTS Hires and Separations Data Using the LDB’ for the details of this method.) The simulation yields estimated employment, hires and separations for those establishments who have entered the labor force for a given month as well as the employment, hires and separations of those establishments that can not be adequately sampled (i.e., establishments less than 12 months old). The simulation also yields estimated separations levels for those establishments who have exited the labor force in a given month.

Birth Employment

The first aspect to be modeled is the level of birth employment (i.e., first time reporters as well as those young firms less than 30 months old) for a given industry for a given month. The birth employment level is taken directly from the monthly simulation of JOLTS data on the LDB. Likewise, the hires and separations levels for the cohort of birth units were taken directly from the simulation.

Death Separations

The separations from the deaths on the LDB were drawn directly from the simulation. Only the first month of each quarter will contain deaths.

Forecasting

Since current LDB data is unavailable when JOLTS estimation is produced, it is not possible to simulate JOLTS birth/death employment, hires and separations. Therefore, it would be necessary to forecast JOLTS birth/death employment, hires and separations. One possible method that can be used to forecast this data would be to use an ARIMA prediction using historical JOLTS birth/death employment, hires and separations data. An ARIMA forecast has been conducted on this data and the forecast performed adequately. It is also possible to forecast using the ratio of CES year ago employment to current employment to adjust birth employment, hires, and separations.

Reference-Attachment:

Addressing JOLTS-CES Divergence

Beginning with the release of January 2009 data on March 10, BLS will implement improvements to the methodology used to generate estimates of hires, separations, and job openings from the Job Openings and Labor Turnover Survey (JOLTS). These changes are designed to improve the measurement of hires, separations, and openings and to more closely align the hires and separations estimates with monthly employment change as measured by the BLS establishment survey.

Research comparing the relationship between JOLTS hires and separations to the monthly employment change measured by the Bureau’s Current Employment Statistics (CES) program (the establishment survey) indicate substantial discrepancies in employment trends over time. While JOLTS does not produce estimates of month-to-month change in employment, an implied employment change can be derived from JOLTS data by subtracting the separations estimate from the hires estimate for a given month. When viewed over time, this derived JOLTS measure of employment change does not track well with the CES, the Bureau’s larger and better-known establishment survey. The CES is designed specifically to measure month-to-month employment change, collects data from a much larger sample, and benchmarks annually to universe employment counts, making CES the more reliable source of monthly employment change. Further, comparison of JOLTS hires and separations data to similar data produced in the Bureau’s Current Population Survey (CPS or household survey) also indicates that JOLTS may be understating the levels of hires and separations.

BLS engaged in a multi-year research project to better understand these two issues, to establish their probable causes, and to develop improvements. As a result of this research, BLS plans to implement improvements in the following areas:

1. Revision of the JOLTS sample design to incorporate new business births more quickly, and to remove business deaths from the frame on a more timely basis;

2. Addition of a birth/death model for JOLTS to provide an estimate of hires, separations, and openings for births which are too new to be captured by the sample and for deaths which often do not get reported during monthly sample collection;

3. Modification to data collection, editing, and review procedures in specific industries where research has indicated a prevalence of particular response errors; and

4. Establishment of a monthly alignment procedure that takes the CES employment change estimates into consideration.

Improvements to the JOLTS Sample Design

Currently, the JOLTS sample is constructed from individual panels of sample units drawn on an annual basis. The full sample consists of one certainty panel made up of large units selected with virtual certainty based on their size, and 24 non-certainty panels. Each year a new set of panels is drawn from the Bureau’s Longitudinal Database (LDB), a product of the Quarterly Census of Employment and Wages (QCEW) program. Each month a new non-certainty panel is rolled into collection, and the oldest non-certainty panel is rolled out. The collection life of a sample panel is therefore 24 months. This means that at any given time the JOLTS sample is constructed from panels from three different sampling frames, the most current being slightly over one year old and the oldest being slightly over three years old. Thus the JOLTS sample design reflects established firms that have been in business for a minimum of one year.

To better reflect the impact of younger establishments in the JOLTS sample, BLS is modifying the JOLTS sample design in the following ways. First, when a new set of panels is selected each year, the birth units in the sample (those not in existence on the previous year’s frame) will be initiated for collection first, rather than waiting until their associated panel is initiated. Second, each quarter the newly updated LDB will be reviewed to identify birth establishments and a supplemental sample of these units will be drawn and added to the survey; at the same time, out-of-business units will be dropped from the sample on a quarterly basis. Thus, the JOLTS sample will be refreshed quarterly rather than annually. Third, the entire sample of old plus new panels will be poststratified and re-weighted annually to represent the most recent sampling frame; at present, this is not done for sample drawn from earlier frames. This procedure will make the sample more efficient than at present.

JOLTS Business Birth/Death Model

As with any sample survey, the JOLTS sample can only be as current as its sampling frame. The sampling frame for JOLTS is drawn from the LDB, which is updated quarterly from files submitted to the BLS QCEW program as part of the State Unemployment Insurance system. The built-in time lag from the birth of an establishment until its appearance on the sampling frame is approximately one year. In addition, many of these new units may fail within the first year. Since these universe units cannot be reflected on the sampling frame immediately, the JOLTS sample cannot capture job openings, hires, and separations from these units during their early existence. To develop data for these units that cannot be measured through sampling, BLS has developed a model to estimate the contribution of these units to the current month estimates. The birth/death model estimates birth/death activity for current month by examining the birth/death activity from previous years on the LDB and projecting forward using the ratio of over-the-year CES employment change. The birth/death model also uses historical JOLTS data to estimate the amount of “churn” (hires plus separations) that exists in establishments of various sizes. The model then combines the estimated churn with the projected employment change to estimate the number of hires and separations taking place in these units that cannot be measured through sampling.

The model-based estimate of total separations is distributed to the three components: quits, layoffs, and other separations, in proportion to their contribution to the sample-based estimate of total separations. Additionally, job openings for the modeled units are estimated by computing the ratio of openings to hires in the collected data and applying that ratio to the modeled hires.

The estimates of job openings, hires, and separations produced by the birth/death modeling process will then be added to the sample-based estimates produced from the survey to arrive at the final estimates for hires, separations, and openings.

Because JOLTS estimates did not previously include this step, addition of the birth/death model will raise the levels and rates of the hires, separations, and openings measured by JOLTS, and allow the series to more accurately reflect the current labor market.

Modifications to Data Collection Procedures

As stated earlier, an implied measure of employment change can be derived from the JOLTS data by subtracting separations from hires for a given month. Aggregating these monthly changes in the current series, however, generally produces employment levels that overstate employment change as measured by CES, at the total nonfarm level. Research into this problem has shown that a significant amount of the divergence between the CES employment levels and the derived JOLTS employment levels can be traced to the Employment Services industry and to the State Government Education industry. In the former industry, businesses have a difficult time reporting hires and separations of temporary help workers. In the latter industry, employers have a difficult time reporting hires and separations of student workers. BLS plans to devote additional resources to the collection, editing, and review of data for these industries. BLS analysts will more closely examine reported data that do not provide a consistent picture over time, and will re-contact the respondents as necessary. Analysts will work with the respondents to adjust their reporting practices as possible. Units that cannot be reconciled but are clearly incorrect on a consistent basis will be dropped from the estimation process and imputed for using existing techniques.

Establishment of an Alignment Procedure

Over time, employment change derived from JOLTS hires minus separations should track well with employment change measured through the CES. However, there are some definitional differences between the series that can cause legitimate differences for individual months. The major reasons for these month-to-month divergences are:

1. The reference periods of the two surveys are different. CES measures employment for the pay period including the 12^th of the month, while JOLTS measures hires and separations for the entire month.

2. CES counts those who worked or received pay for the reference pay period, while JOLTS counts those who were hired or separated during the reference month. It is possible for a person to miss being paid for a given pay period without having been separated.

Both of these definitional differences can result in differing seasonal patterns between the two series, and therefore cause JOLTS to diverge from the CES in the short-term. Over time however, the computation of JOLTS hires minus separations should reflect employment changes that are consistent with the trends measured by the CES. The three changes to JOLTS that have been described above are expected to produce JOLTS series’ that are much more consistent with the CES. The residual divergence will be controlled through a monthly alignment procedure that allows JOLTS to vary from CES for the reasons listed above, while ensuring that the long-term trends in JOLTS hires-minus-separations match those of the CES net employment change.

The goal of this process is to use current monthly CES employment trends to align the JOLTS implied employment trend (hires minus separations) to be approximately the same, but without forcing all the seasonal patterns to be the same between the surveys. This method takes advantage of the fact that the CES employment series for the current reference month is available prior to the production of JOLTS estimates for that same reference month.

The method works as follows:

• Each month, the initially computed seasonally adjusted JOLTS hires-minus-separations employment change estimate is adjusted to equal the CES seasonally adjusted net employment change estimate, through a proportional adjustment of the hires and separations estimates. By comparing the JOLTS and CES seasonally adjusted changes, the alignment procedure preserves legitimate differences in the seasonal patterns of underlying JOLTS and CES

• Proportional adjustment means that the two components (hires, separations) are adjusted in proportion to their contribution to the total churn (hires plus separations). For example, if hires is 40% of the churn for a given month, it will receive 40% of the needed adjustment and separations will receive 60% of the needed adjustment.

• In the next step, these adjusted hires and separations estimates are converted back to not seasonally adjusted data by reversing the application of the original seasonal adjustment factors.

• These trend-corrected not seasonally adjusted series are then put through the standard X-12 ARIMA seasonal adjustment process to create the final seasonally adjusted published series. These final seasonally adjusted series will not precisely equal the CES seasonally adjusted net employment change but will be very similar.

Revisions to Historical Series

The monthly JOLTS series begin with estimates for December 2000. All published estimates back to that point will be revised to reflect the addition of the revised birth-death model and the new alignment procedure, as well as selected adjustments to individual survey reports. New historical series for job openings, hires, total separations, quits, layoffs and discharges, and other separations will replace the currently available series. At that time, tables comparing the original and revised series will also be available.

0 This data is representative of a cohort of establishments which had only been in on the LDB for 12 months or less. These are precisely the establishments that the JOLTS survey can not sample, enroll and collect data from.

i

11