Part B 2008 Supporting Statement National Compensation Survey

B. Collection of Information Employing Statistical Methods

Planning for the National Compensation Survey (NCS) involved the consideration of alternative designs within the overall budgetary constraint. Some of the major elements entering into these considerations were the basic products desired, the availability of data, and requirements to assure statistically reliable estimates. Other elements considered were the efficiency of alternative collection procedures and the probable degree of cooperation from respondents.

1. Respondent Universe and Response Rate

The potential respondent universe used in the selection of the NCS sample of establishments is derived from the Quarterly Census of Employment and Wages (QCEW) and a supplementary file of railroads for each area in the sample. The QCEW is created from State Unemployment Insurance (UI) files of establishments, which are obtained through the cooperation of the individual State agencies. UI accounts are assigned to all employers in the United States who are required to pay for unemployment insurance. The NCS universe includes all State and local governments and private sector industries, except for farms and private households. Estimates of the current universe and sample size of all Primary Sampling Units combined are about 6,368,000 and 34,700 establishments, respectively. Data on the duties and responsibilities of occupations will be collected in all sample establishments.

NCS’s projected 34,700 sample establishments, approximately 16,800 establishments will have quarterly collection for the employment costs and benefits participation and plans provisions. The approximately 17,900 remaining establishments will have annual collection of earnings data to produce locality and national data. These numbers refer to the total sample the NCS will build up to; the tables in the previous section show smaller amounts, due to assumed loss because of refusals, companies going out of business, etc.

The response rate is expected to be about 78 percent for earnings only initiation schedules and 74 percent for earnings and benefits initiation schedules. Response rates for update of earnings only schedules among schedules that responded at initiation is estimated at 93 percent, and 90 percent for earnings and benefits update schedules.

2. Sample Design

Stratification and Sample Selection

The NCS sample is selected using a 3-stage stratified design with probability proportional to employment sampling at each stage. The first stage of sample selection is a probability sample of areas, the second stage is a probability sample of establishments within sampled areas, and the third stage of sample selection is a probability sample of occupations within sampled areas and establishments. For more information on the sample design, see the Bureau of Labor Statistics Handbook of Methods, Chapter 8, available on the BLS Internet. http://www.bls.gov/opub/hom/homch8_a.htm

The selection of sample areas is done by first dividing the entire area of the United States, consisting of counties (or county equivalents such as parishes in Louisiana) and independent cities, into primary sampling units (PSUs). In most States, a PSU consists of a county or a number of contiguous counties. Metropolitan and micropolitan areas, as defined by OMB, are used as a basis for forming PSUs. Outside of metropolitan and micropolitan areas, a cluster of contiguous counties defines a PSU.

The PSUs with similar average earnings are grouped into strata within each of the 9 census divisions and three area types (Metropolitan, Micropolitan, outside of metropolitan and micropolitan). One PSU is selected from each stratum with the probability of selection proportional to the employment of the PSU. There are 57 PSUs in strata by themselves that are self-representing, and these include the 27 Combined Statistical Areas (CSAs), the 29 largest Metropolitan Statistical Areas (MSAs), and 1 additional metropolitan area needed to meet the needs of the Pay Agent. The remaining strata are formed by combining PSUs that are MSAs and have similar average annual pay into 60 MSA strata, PSUs that are Micropolitan areas and have similar annual average pay into 22 Micropolitan strata, and PSUs that are outside of metropolitan and micropolitan areas and have similar average annual pay into 13 strata. The PSUs selected with probability proportionate to PSU employment from these strata are non-self-representing because each one chosen represents the entire stratum.

The NCS program started transitioning to the Metropolitan Statistical Areas, Metropolitan Divisions, Micropolitan Statistical Areas, and Combined Statistical Areas in the United States based on the standards published on December 27, 2000, in the Federal Register (65 FR 82228 - 82238) in FY 2007. Current Lists of Metropolitan and Micropolitan Statistical Areas and definitions is at this link: http://www.census.gov/population/www/estimates/metrodef.html

Each sample of establishments is drawn by first stratifying the sampling frame for each PSU by industry and ownership. The strata for private industry, local government, and State government (North American Industry Classification System -- NAICS based) are as follows:

NCS Stratification

Private Industry

NAICS Industry

21 Mining

22 Utilities

23 Construction

31-33 (excl 336411) Manufacturing

336411 Aircraft Manufacturing

42 Wholesale Trade

44-45 Retail Trade (rest of)

48-49 Transportation and Warehousing

51 Information (rest of)

52 (excl 524) Finance (excluding Insurance)

524 Insurance Carriers and Related Activities

53 Real Estate and Rental and Leasing

54 Professional, Scientific, and Technical Services

55 Management of Companies and Enterprises

56 Administrative and Support, Waste Management

61(excl 6111-61130) Education (rest of)

6111 Elementary & Secondary Education

6112, 6113 Colleges & Universities

62 (excl 622,623) Health and Social Assistance (rest of)

622 Hospitals

623 Nursing Homes

71 Arts, Entertainment, and Recreation

72 Accommodation and Food Services

81(excl 814) Other services except public administration

Local Government

NAICS Industry

21, 23, 31-33 Goods-Producing

42, 44-45, 48-49, 22 Trade, Transportation, and Utilities

6111 Elementary & Secondary Education

6112, 6113 Colleges and Universities

61 excl. 6111-6113 Rest of Education

622 Hospitals

623 Nursing Homes

62 excl. 622-623 Rest of Health and Social Services

92 excl. 928 Public Administration

51,52-53,54-56,71-72,81 excl 814 Other Service -producing

State Government

NAICS Industry

21, 23, 31-33 Goods-Producing

42, 44-45, 48-49, 22 Trade, Transportation, and Utilities

6111 Elementary & Secondary Education

6112, 6113 Colleges and Universities

61 excl. 6111-6113 Rest of Education

622 Hospitals

623 Nursing Homes

62 excl. 622-623 Rest of Health and Social Services

92 excl. 928 Public Administration

51,52-53,54-56,71-72,81 excl 814 Other Service -producing

The number of sample establishments allocated to each stratum is approximately proportional to the stratum employment. Each sampled establishment is selected within a stratum with a probability proportional to its employment. The allocation process used in the NCS is documented in the 2005 ASA Proceedings of the Survey Research Methods Section¹.

After the sample of establishments is drawn, occupations are selected in each sampled establishment. The number of occupations selected in an establishment ranges from 4 to 8 depending on the total number of employees in the establishment, except for government and aircraft manufacturing units and units with less than 4 workers. In governments, the number of occupations selected ranges from 4 to 20. In aircraft manufacturing, the number of occupations selected ranges from 4 for establishments with less than 50 workers to 32 for establishments with 10,000 or more workers. In establishments with less than 4 workers, the number of occupations selected equals the number of workers. The probability of an occupation being selected is proportionate to its employment within the establishment.

Estimation Procedure

The survey will produce level estimates, such as average earnings of professional workers at the entry level, along with quartiles, first and last deciles, and indexes. The estimation procedures for the earnings and index estimates are described below. The Index procedure also includes seasonal adjustment. Note that both of these procedures involve weighting the data from each employee in the sampled occupation by the final weight.

The final weights include the initial sample weights, adjustments to the initial sample weights, three types of adjustments for non-response, and benchmarking. The initial sample weight for an occupation in a particular establishment and PSU reflects the probability of selecting a particular PSU, the probability of selecting a particular establishment within the PSU, and the probability of selecting a particular occupation within the selected establishment and PSU. Adjustments to the initial weights are done when data are collected for more or less than the sampled establishment. This may be due to establishment mergers, or splits, or the inability of respondents to provide the requested data for the sampled establishment. The two types of adjustments for non-response include adjustment for establishment refusal to participate in the survey and adjustment for respondent refusal to provide data for a particular occupation.

Benchmarking or post-stratification is the process of adjusting the weight of each establishment in the survey to match the distribution of employment by industry at the reference period. Because the sample of establishments used to collect NCS data was chosen over the past several years, establishment weights reflect their employment when selected. For outputs other than the ECI, the benchmark process updates that weight based on current employment. For the ECI, the benchmark process updates that weight based on the employment during the publication base period.

The estimation procedure for level estimates, such as mean weekly earnings, mean annual wages, and mean hourly earnings, use the individual weight of the sampled occupation, the individual rates in the sampled occupations, and the number of weeks worked per year. The calculation of the average hourly wage also includes the number of hours paid per week. For mean weekly earnings this involves multiplying the weekly wage rate for each employee in the sample occupation by the final weight and the number of annual weeks worked, summing, and dividing by the sum of the final weights times the number of workers for which the NCS collected data times the number of weeks worked. See pages 6 and 7 in Chapter 8 of the BLS Handbook of Methods (available on the BLS Internet at http://www.bls.gov/opub/hom/homch8_a.htm) for an explanation of the estimation procedures for Employer Costs for Employee Compensation estimates and for Benefit Incidence and Provisions estimates.

The index computation involves the standard formula for Laspeyres fixed-employment-weighted index, modified by the special statistical conditions that apply to the NCS. An index for a benefit derived from the NCS data is simply a weighted average of the cumulative average benefit costs changes within each estimation cell, with base-period benefit bills as the fixed weights for each cell. This discussion focuses on the ECI measures of benefit cost changes, but indexes of changes in compensation and wages are computed in essentially the same fashion.

The simplified formula is:

Numerator =

Denominator =

where:

i = estimation cell

t = time

is the index at time t

is the estimated base-period benefit bill for the ith estimation cell. The benefit bill is the average benefit cost of workers in the cell times the number of workers represented by the cell.

is the cumulative average benefit cost

change in the ith estimation cell from time 0 (base period) to time t (current quarter).

is the cumulative average benefit cost change in the ith estimation cell from time 0 (base period) to time t-1 (prior quarter).

is the ratio of the current quarter weighted average benefit cost in the cell to the prior quarter weighted average benefit cost in the cell, both calculated in the current quarter using matched establishment/occupation observations.

The estimation cell is defined on the basis of ownership/industry/major occupation group. For the public sector, separate cells are identified for State and for local governments. Industries as broad as “public administration” and as narrow as “colleges and universities” are treated as separate estimation cell industries. For example, one estimation cell is identified as State government/public administration/clerical workers.

The index computations for the occupation and industry groups follow the same procedures as those for all overall indexes except for the summation. The bills for the occupational groups are summed across industries for each group; the bills for the industry divisions are summed across occupational groups for each industry division.

Computational procedures for the regional, union/nonunion, and metropolitan/non-metropolitan measures of change differ from those of the “national” indexes because the current sample is not large enough to hold constant the benefits bills at the level of detail. For these “non-national” series, each quarter the prevailing distribution in the sample between, for example, union and nonunion within each industry/occupation cell, is used to apportion the prior quarter benefits bill in that cell between the union and nonunion series. The portion of the benefits bill assigned to the union sector is then moved by the percentage change in the union earnings in the cell, and similarly for the nonunion sector. Thus, the relative importance of the union sector in each cell is not held constant over time. Since the relative weights of the region, the union, and the metropolitan area sub-cells are allowed to vary over time, the non-national series are not fixed base period Laspeyres indexes; rather, these are similar to chain linked Laspeyres indexes.

Seasonal Adjustment

Index series with 8 to 10 years of data can be considered for seasonal adjustment, our seasonally adjusted index series are currently fixed. The X-12 ARIMA (Auto-Regressive Integrated Moving Average) procedure, which was developed at the U.S. Bureau of the Census as an extension of the standard X-11 method, will be used to perform seasonal adjustment. At the beginning of each calendar year, seasonal adjustment factors are calculated for use during the coming year. The factors for the coming year are posted on the BLS website. Revisions of historical seasonally adjusted data for the most recent five years also appear in the CWC.

The ECI series are seasonally adjusted using either direct or composite estimates. Most industry and occupational series such as construction, for example, are adjusted directly. The civilian, State and local government, private, and manufacturing series are composite estimates. The seasonally adjusted civilian benefits series, for example, is computed by aggregating the following independently adjusted series: private goods-producing benefits, private service-producing benefits, and State and local government benefits.

Sample Replacement Scheme

NCS selects a new sample of areas approximately once every ten years. The current sample of areas was selected from areas that were defined by OMB after the 1990 census. In 2004, NCS selected a new sample of areas from areas that were based on the 2000 census. Each year, NCS selects a new sample of establishments from the most recent available frame data. Private industry establishments from the old area sample will be rotated out of the NCS survey outputs over the next 6 years as the new area private industry establishment samples are rotated into the survey outputs. Beginning in 2006, a new sample of State and local government establishments was selected and will be introduced into NCS at the end of 2007. The State and local government establishments will remain in the sample at least 6 years. A new sample of occupations is selected within each establishment at least once every 6 years as the establishment is initiated into the survey process. Under this scheme the entire NCS sample is completely replaced every 6 years.

The primary objectives of the replacement scheme are to reduce reporting burden of individual establishments by rotating units out of the sample and to insure that the establishment sample is representative of the universe it is designed to cover over time.

Measuring the Quality of the Estimates

The two basic sources of error in the estimates are bias and variance. Bias is the amount by which estimates systematically do not reflect the characteristics of the entire population. Many of the components of bias can be categorized as either response or non-response bias.

Response bias occurs when respondents’ answers systematically differ, in the same direction, from the correct values. For example, this occurs when respondents incorrectly indicate no change in benefits costs when benefits costs actually increased. Another possibility of having response bias is when data are collected for a unit other than the sampled unit. Response bias can be measured by using a re-interview survey. Properly designed and implemented, this can also indicate where improvements are needed and how to make these improvements. The NCS has a Technical Re-interview Program (TRP) that does a records check of a sample of each field economist’s schedules of collected data. TRP is a part of the overall review process. TRP verifies directly with respondents a sample of elements originally collected by the field economist. The results are reviewed for adherence to NCS collection procedures. This program allows the NCS to identify procedures that are being misunderstood and to make improvements in the NCS Data Collection Manual and training program.

Non-response bias is the amount by which estimates obtained do not properly reflect the characteristics of non-respondents. This bias occurs when non-responding establishments have earnings and benefit levels and movements that are different from those of responding establishments. Non-response bias is being addressed by continuous efforts to reduce the amount of non-response. NCS is analyzing the extent of non-response bias using administrative data from the survey frame. The results from initial analysis are documented in the 2006 ASA Proceedings of Survey Research Methods Section². Further analysis is on-going.

Another source of error in the estimates is sampling variance. Sampling variance is a measure of the fluctuation between estimates from different samples using the same sample design. Sampling variance in the NCS is calculated using a technique called balanced half-sample replication. For national estimates this is done by forming 128 different re-groupings of half of the sample units. For each half-sample, a "replicate" estimate is computed with the same formula for the regular or "full-sample" estimate, except that the final weights are adjusted. If a unit is in the half-sample, its weight is multiplied by (2-k); if not, its weight is multiplied by k. For all NCS publications, k = 0.5, so the multipliers are 1.5 and 0.5. Sampling variance computed using this approach is the sum of the squared difference between each replicate estimate and the full sample estimate averaged over the number of replicates and adjusted by the factor of 1/(1-k)² to account for the adjustment to the final weights. For more details, see the NCS Chapter of the BLS Handbook of Methods. Standard error, which is the square root of variance, for primary aggregate estimates of the index of quarterly change are typically less than 0.5 percent. Relative standard error, which is the square root of variance divided by the estimate, for aggregate estimates of compensation, wage, or benefit levels are typically less than 5 percent. As appropriate, the standard errors or relative standard errors are included within published NCS reports.

Variance estimation also serves another purpose. It identifies industries and occupations that contribute substantial portions of the sampling variance. Allocating more sample to these domains often improves the efficiency of the sample. These variances will be considered in allocation and selection of the future replacement samples.

3. Non-response

There are three types of non-response: permanent non-response, temporary non-response, and partial non-response. The non-responses can occur at the establishment level, occupation level, or benefit item level. The assumption for all non-response adjustments is that non-respondents are similar to respondents.

To adjust for permanent establishment or occupation non-response at the initial interview, weights of responding units or occupations that are deemed to be similar are adjusted appropriately. Establishments are considered similar if they are in the same 2-digit NAICS. If there are no sufficient data at this level, then a broader level of aggregation is considered.

For temporary and partial non-response, a replacement value is imputed based on information provided by establishments with similar characteristics. Imputation is done separately for each benefit both in the initial period and in subsequent update periods. Imputation is also done for each missing wage estimate after the initial period. In the rare event that the BLS cannot determine whether or not a benefit practice exists for a non-respondent, the average cost is imputed based on data from all responding establishments (including those with no plans and plans with zero costs).

There is a continuous effort to maximize response rate. We are developing and providing respondents with new and useful products. We are looking into providing alternative methods for respondents to report their data.

4. Test of Procedures

Further testing of a web page based data collection system will continue, with implementation following.

5. Responsibilities

The Bureau of Labor Statistics, Office of Compensation and Working Conditions, Statistical Methods Group has the responsibility for the statistical aspects of the NCS design. Gwyn R. Ferguson (202-691-6941) is the contact regarding statistical issues for the survey. The contact regarding collection and analysis is Mr. Hilery Simpson (202-691-5184).

1 Yoel Izsak, Lawrence R. Ernst, Erin McNulty, Steven P. Paben, Chester H. Ponikowski, Glenn Springer, Jason Tehonica, "Update on the Redesign of the National Compensation Survey", 2005 Proceedings of the American Statistical Association, Section on Survey Methods Research [CD-ROM], American Statistical Association, 2005 http://www.bls.gov/ore/pdf/st050140.pdf

2 Ponikowski, Chester H. and McNulty, Erin E., " Use of Administrative Data to Explore Effect of Establishment Nonresponse Adjustment on the National Compensation Survey", 2006 Proceedings of the American Statistical Association, Section on Survey Methods Research [CD-ROM], American Statistical Association, 2006

http://www.bls.gov/ore/abstract/st/st060050.htm

10