Supporting_Statement_Part_B_QBS_Operations_Test_2_2019

Quick Business Survey

Operations Test 2

1220-0192

March 2019

Second Operations Test of the Proposed Quick Business Survey

Part B. Statistical Methods

1. Respondent/Sample Universe

1a. Universe

The Bureau of Labor Statistics (BLS) plans to conduct a second operations test of the proposed Quick Business Survey (QBS) on a sample of establishments included in the BLS Quarterly Census of Employment and Wages (QCEW). The universe of respondents to the QCEW are the 50 States, the District of Columbia, Puerto Rico, and the U.S. Virgin Islands. The primary source of data for these 53 entities are the Quarterly Contribution Reports (QCRs) submitted to State Workforce Agencies (SWAs) by employers subject to State Unemployment Insurance (UI) laws. The QCEW data, which are compiled for each calendar quarter, provide a comprehensive business name and address file with employment and wage information by industry, at the six-digit North American Industry Classification System (NAICS) level, and at the national, State, Metropolitan Statistical Area (MSA), and county levels for employers subject to State UI laws. Similar data for Federal Government employees covered by the Unemployment Compensation for Federal Employees program (UCFE) also are included.

The scope of the Quick Business Survey is:

• Establishments in-scope of the Annual Refiling Survey (ARS).

• Establishments with one to three employees that are excluded from the ARS.

• Establishments that are in low-change NAICS industries that are excluded from the ARS.

The QBS sample will be restricted to those eligible for the 2020 ARS using the 7^th and 8^th digit of the EIN within the range of 67-99 from the QCEW. The QBS sampling frame of approximately 2.2 million establishments includes this 1/3 of the ARS, plus all other in-scope establishments within this EIN range.

1b. Sample size

BLS will attempt to leverage online response to the ARS, its largest annual survey, by asking respondents to answer additional questions after they have completed the survey. The size of the ARS, with over one million sample units, requires a collection timeframe of 9-11 months. A random sample for a QBS supplement to the ARS, regardless of size, would require very little additional cost to collect. However, this timeframe would hardly be considered “quick.” BLS would like to take advantage of the high-volume of reporting that occurs immediately following contact with respondents, either through email solicitations or paper mailings, by sampling exclusively during these peak response times. The volume of these responses, and the distribution of these responses during peak times across sample strata, is uniform enough to suggest that data for a representative sample can be collected in a matter of weeks, not months. BLS plans to select three samples for this operations test, for a total of 44,062 sample units and an estimated 26,826 responses.

Sample 1

A stratified random sample of 15,000 establishments will be selected from the QBS in-scope units described in Section 1a. Based on the results of the first test, BLS expects about 60 percent of the random sample will complete the ARS, and 79 percent of the random sample respondents that complete the ARS and are asked to answer further questions online will “click-through” to see the additional questions. BLS further expects about 96 percent of respondents that click-through will answer the questions online and submit their responses.

Sample 1 Size for QBS Operations Test
	Sample	ARS Responses (60%)	Click-through Respondents (79%)	Estimated Responses (96%)
Random Sample	15,000	9,000	7,110	6,826

Sample 2

A second sample will be collected to obtain 10,000 QBS responses from the remaining ARS respondents.

Sample 2 respondents will not be selected in advance. After completing the ARS, their QBS eligibility will be determined based on the number of completed responses in their industry-size class cell during that period. For example, we will aim to collect a given quantity of responses from Construction units (NAICS = 23), of size class 3 (10- to-19 employees). ARS respondent meeting that criteria will be randomly offered the QBS until the desired level of response for that cell is achieved. The industry (at the 2-digit NAICS level), size class, and time conditions have been specified with the goal of maximizing representation of this sample while collecting QBS data as quickly as possible. For the stratified quota samples (Sample 2 and Sample 3), BLS will attempt to collect 10,000 responses. From the results of the first test BLS expects that 74 percent of the ARS respondents offered the QBS will click through to see the QBS questions, and 93 percent will complete the QBS and submit their responses.

Sample 2 Size for QBS Operations Test
	Sample	ARS Responses	Click-through Respondents (74%)	Estimated Responses (93%)
Stratified Quota Sample	14,531	14,531	10,753	10,000

Sample 3

A third sample will be collected to obtain another 10,000 QBS respondents. Sample 3 will be collected similarly to Sample 2 with one specific difference. Sample 3 will be collected after completion of Sample 2, over the remaining ARS collection period. This will be done to test the difference between early (i.e. quick) responders and all other responders. The industry (at the 2-digit NAICS level), size class, and time conditions have been specified with the goal of maximizing representation of this sample. Sample 3 will be used to determine whether bias exists in collecting data quickly (Sample 2).

Sample 3 Size for QBS Operations Test
	Sample	ARS Responses	Click-through Respondents (74%)	Estimated Responses (93%)
Stratified Quota Sample	14,531	14,531	10,753	10,000

2. Data Collection

2a. Sample Design and Selection Procedures

The random sample of 15,000 establishments (Sample 1) will be allocated proportionally to the strata (with a minimum sample of 5 units in each strata). The strata are defined by multi-unit/single-unit employer, 2-digit NAICS industry, and 9 size classes. The sample will be selected randomly within each stratum. All private sector industries except private households are in-scope for this test. The nine size classes are determined by the establishment’s Average Monthly Employment (AME) over the last 12 months -- employment 1-4 (size 1); 5-9 (size 2); 10-19 (size 3); 20-49 (size 4); 50-99 (size 5); 100-249 (size 6); 250-499 (size 7); 500-999 (size 8); and 1000+ (size 9).

In addition, per the description in Section 1b, the stratification of the quota samples will mirror the random sample, and the quota sample respondents will be analyzed to determine if they can be used to make representative estimates.

2b. Estimation Procedure

Because this operations test is experimental and in its initial phase, no estimates will be computed.

2c. Reliability

This operations test is experimental and in its initial phase no sampling errors will be computed.

2d. Data Collection Cycles

This survey will be conducted one time, from July 2019 through June 2020.

3. Response Rates

3a. Maximize Response Rates

To maximize response rates, all units selected in the sample will be transitioned to the QBS questions once they complete the ARS. All multi-units and any singles that are out of scope for the ARS, but are selected for the QBS, will be contacted using established ARS contact methods, i.e. email and ARS solicitation letters.

Non response follow-up will be conducted per the current ARS procedures as outlined in materials for OMB Control No. 1220-0032.

3b. Non-Response Adjustment

Because this operations test is experimental and in its initial phase no adjustment will be calculated.

4. Testing Procedures

Upon completion of the operations test we will calculate “click-through” rates, QBS completion rates, and non-response rates. The click-through rate is the percentage of ARS respondents that click on the button after completing the ARS to respond to the additional survey questions. This would be similar to calculating the number of respondents that open the envelope when sent a survey solicitation in the mail.

The QBS completion rate is the percentage of respondents that complete the QBS survey online, and the non-response rate is the percentage of sample units that do not submit their data online.

5. Statistical and Analytical Responsibility

Mr. Ed Robison, Division Chief of the Statistical Methods Staff, Office of Employment and Unemployment Statistics, is responsible for the statistical aspects of this survey.

5