1910-5179- USEER Supporting Statement B

Supporting Statement for United States Energy and Employment Report Data Collection

1. Part B: Collections of Information Employing Statistical Methods

OMB No. 1910-5179

August 2024

U.S. Department of Energy

Washington, DC 20585

Table of Contents

Part B: Collections of Information Employing Statistical Methods i

B.1. Respondent Universe 1

B.2. Statistical Methods 2

B.3. Maximizing Response Rates 3

B.4. Test Procedures and Form Consultations 6

B.5. Statistical Consultations 6

B.1. Respondent Universe

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used.

Geographic coverage includes the 50 States, the District of Columbia, and Territories. Private establishments and government units are included, but establishments must report at least one permanent employee to be included. Data are to be collected for establishments in 150 detailed industries identified to be of specific interest for the USEER (United States Energy & Employment Jobs Report) Survey. The industries are defined using the 6-digit detail of the North American Industry Classification System (NAICS; includes 1,012 6-digit industries). Attached is a table summarizing the survey frame the industry sector level (Section B attachment).

The sampling frame is a representative sample of employers based on establishment totals from the Quarterly Census of Employment and Wages (QCEW) Longitudinal Database (LDB) maintained by the Bureau of Labor Statistics, stratified by employment size categories developed by the Census Bureau County Business Patterns data set. The actual contact information and business names are drawn from a private dataset, DataAxleUSA, because the QCEW is confidential. About 3,080,696 establishments with employment of 26.4 million are in the 166 in-scope industries (4 to 6-digit NAICS industries). Due to clustering by size, a total of 1,872,000 establishments are included in the sampling frame.

For the purposes of USEER sample allocation, we aggregate 166 detailed industries into 7 groups of industries or “allocation” NAICS (ANAICS). For most in-scope industries, the ANAICS is the 2-digit NAICS and includes all in-scope NAICS-defined industries within the 2 digit. Within some 2-digit industries, ANAICS splits out specific 5- and 6-digit NAICS industries where we anticipate having a higher incidence of energy activity. ANAICS 2- and 3-digit coding is the same as for NAICS, though restricted to EJR(Energy Jobs Report)-eligible industries.

Industry sectors are also defined for use in allocation. Industry sectors are 2-digit ANAICS with two exceptions. The manufacturing sector combines three 2-digit codes. The trade sector combines retail trade and wholesale trade.

About 11,200 in-scope “Known Universe” establishments were pre-identified as having “energy” activity. A database of likely energy establishments will be developed internally by Contractor by collecting industry association databases, approved utility contractor lists, and other public and private sources. By comparing the information obtained through these sources and comparing the NAICS codes of these establishments on the QCEW, Known Universe establishments will be matched to the QCEW/DataAxleUSA dataset and a “known” indicator will be used to assist in oversampling “known” establishments.

Contractor will contact about 42,000 establishments per year. The total survey completion targets will be based on a sample selected using the QCEW/DataAxleUSA frame for the second quarter of 2024. Quotas will be established for each NAICS code or ANAICS code by size. Although the precision is not specified, the goal is to publish industry sector data in every State and to obtain useful data at the national and state level for each ANAICS and identified technology area. If possible within budget constraints, some additional information by 6-digit NAICS is also desired. USEER establishments are not separated but are included in columns for the private, Federal government, State government, and local government sectors.

Stratification – The USEER will be stratified by 6-digit NAICS or ANAICS and size class (1-9, 10-19, 20-49, 50-99, and 100+ employees) and systematic samples selected in the noncertainty strata. Known establishments can be of any ownership, are processed separately, and are excluded from the other portions of the frame. Federal government stratification is State by industry sector. State government stratification is State by industry sector. For private establishments (excluding the Known Universe) three levels of stratification are examined during sample allocation: 1) State x industry sector, 2) national ANAICS, and 3) national 6-digit NAICS.

B.2. Statistical Methods

Describe the procedures for the collection of information including:

USEER panels will have a probability-based sample aimed at satisfying data needs at both the State x industry sector level and the national ANAICS level. The basic sampling unit is an establishment. Response quotas will be established based on the representation of total establishments by 6-digit NAICS, times the proportion of establishments in each size category as identified in the most recent available data from Census Bureau County Business Patterns.

Restricted to in-scope industries, establishment on the QCEW frame are separated into 4 mutually exclusive parts that are separately sampled. Approximate sample counts refer to a sample selected from the QCEW frame for quarter 2 of 2024.

• Known Universe; census, with up to six attempts; stratification industry by size class (can have any ownership code)

• Federal Government; sample 50; stratification state by industry sector

• State Government; sample 50; stratification state by industry sector

• Private; sample 15,000; complex stratification using state, industry

Known Sampling – All establishments in the Known Universe will be contacted up to six times. The responses will be treated separately, and the overall employment from the Known Universe sample will be deduplicated from the appropriate panel of ANAICS, based on the Known Universe respondent NAICS code.

Private Establishments and Government (excluding Known Universe) –The allocation has 4 basic steps.

• Establishments by State – relying on the most recent data available from QCEW, Contractor will determine the proportion of establishments in each selected NAICS, as a percentage of the total establishments in all selected NAICS.

• NAICS Establishments by Size– relying on the most recent data available in the Census Bureau’s County Business Patterns, Contractor will determine the proportion of establishments within each size category in each 6-digit NAICS. The total NAICS quota will then be allocated by the size proportions to develop the percentage of total state-level sample.

• Deduplicate Known Universe Establishments from Sampling Universe – verifying by name, NAICS, contact name, address, phone, and other identifying information, Known Universe establishments will be removed from the private, state, and federal government sampling universes.

• Establish Quotas – State-level quotas are established by multiplying the total number of proposed survey completions per state by the percentage established in “Establishments by State” above, and by the percentage established in “NAICS Establishments by Size” above.

Changes in the Sample Design – When EJR data become available, it will be possible to more efficiently design the sample to meet targeted data needs. It is anticipated that changes in industry scope may be made.

EJR estimation requires a multi- step process. First, a rate is calculated for qualifying firms based on their response relative to conducting qualifying energy activities. This is referred to as the incidence rate and is used to identify the total number of QCEW establishments, by NAICS and State, that conduct energy activities. The second step of the process is to determine the average share of employment in qualifying establishments that are focused on energy-related portions of the business. This is called the energy share rate.

The incidence rate is then multiplied by the Known Universe establishment total, after reduction for churn. The total employment of these qualifying firms is then removed from the QCEW total by corresponding NAICS code (employments and establishments). The total employment of these qualifying firms is also multiplied by the energy share rate to determine energy employment in the Known Universe.

Private and Government (Known excluded) employment is calculated by multiplying the sum of QCEW employment by NAICS code, less Known Universe Employment by NAICS code, times incidence rate by NAICS code, times energy share by ANAICS code.

Total energy employment is calculated by adding Private, Government, and Known energy employment.

Value chain employment is allocated by NAICS industry. Technology employment is allocated by developing a “technology share” percentage in the instrument, by ANAICS.

B.3. Maximizing Response Rates

Describe methods to maximize response rates and to deal with issues of non-response.

Respondents to the survey will have the option to opt-in to a drawing for one of 20 $100 Visa Gift Cards. Gift cards will be offered to employers that complete the full survey and indicate their willingness to share an email address. Gift cards will be distributed via email to a random selection of 20 employers that completed the full survey at the conclusion of data collection.

To mitigate possible bias arising from nonresponse, weighting class adjustments to the weights will be made. The process for determining nonresponse weight factors, will differ, depending on whether we are adjusting the known or unknown sample.

Weighting for nonresponse in the known sample

Step 1: Quantifying the Known Universe of Energy Firms

The known universe of energy firms will be reversed matched, based on firm name and available contact information (phone number and address) against DATAAXLEUSA’s national database of businesses. This reverse match will provide a consistent designation by traditional industry code (4 to 6 digit NAICS code) and an initial estimate of employment size by firm location. In our experience the reverse match, will identify 65 to 80 percent of known energy firms. The remaining 35 to 20 percent of known firms that are not matched with DATAAXLEUSA’s national database will be evaluated individually and using publicly available data, we will estimate NAICS industry for each firm and an estimate of employment size by location. The portion of known firms that cannot be matched to DATAAXLEUSA’s national database or an estimate of industry classification, location information and when available employment by location¹ made will be discarded from the known universe. This analysis will provide a universe of known energy firms that are categorized by industry classification (NAICS code), location² (State and zip code), and employment by location³ size.

Step 2: Stratifying the known sample by industry (NAICS), Geography, and Employment size by location

The sampling plan for the known universe will be a census approach, with the objective of maximizing participation among all potential participants. Completed surveys will be categorized by industry, location, and employment size as is consistent with the known universe. The known universe will also be updated to reflect businesses that have gone out of business or are no longer involved in energy related work or whose original assumptions about industry, location and employment size identified in step 1 have been revised. This second step will allow the known sample to be compared against the known universe among energy firms.

Step 3: Comparing the known universe to the known sample and producing adjusted weights

The adjusted weights will be based on the relationship between all energy employment in the known universe and total employment in the known universe for all energy firms in each stratum (i.e. each stratum will be based on Industry (I), Location (L) and Size of firm (S)) and all energy employment in the known sample and total employment in the known sample for each stratum.

Known Universe Energy Employment (ALL FIRMS) = ∑ KEN AF

Known Sample Employment (ALL FIRMS) = ∑ Ken AF

Each stratum’s total known universe employment = ∑ KEN ILS

Each stratum’s total known sample employment = ∑ Ken ILS

Weight for each stratum within the known sample = (∑ KEN ILS / ∑ KEN AF ) / (∑ Ken ILS / ∑ Ken AF )

The weight for each stratum is than applied to each completed survey that falls into that stratum definition.

Weighting for nonresponse in the unknown sample

The unknown sampling plan is typically informed by what was learned in quantifying the known universe of energy firms. It provides some insight into what industries, areas and sizes of businesses energy businesses are more likely to be found. The analysis of the known universe along with a literature review of new and emerging energy industries, provides the basis for the traditional NAICS codes that are to be examined and sampled within the unknown universe.

Step 1: Removing known energy businesses from the unknown sample

The known universe of energy firms will be removed from the relevant NAICS codes that are being sampled in the unknown sample. This will create a revised unknown universe (Total NAICS code universe – known Energy Firm NAICS universe) in each of the NAICS codes on interest to be examined in the unknown sample, and ensure that employment is not double counted between the known and unknown universes. Estimate for unknown employment and establishments by Industry, Location and Size will be based on QCEW, CENSTATS and a proportional subtraction based on revised unknown employment and establishments.

Known = ∑ employment/establishments within a given NAICS code based on Known Universe Energy employment

Total = ∑ employment/establishments within a given NAICS code based on CENSTAT’s County Business Patterns estimate

Revised Unknown Universe Employment by Industry of Interest = Industry Employment Estimate⁴ * (1 – (Known/Total))

Revised Unknown Universe Establishment count by Industry of Interest = Industry Establishment Estimate⁵ * (1 – (Known/Total))

Step 2: Develop a sampling plan based on the Industry (NAICS) categories of interest and develop stratum of sampling by Industry, Location and Size of Firm (ILS) based on Revised Unknown Universe of Interest

The revised unknown universe for each industry of interest will serve as the basis for the sampling plan and the number of establishments and total employment within each stratum. Stratum are again based on Industry, Location and Size of the firm. The sampling plan will determine the size of the sample that is needed for each stratum of the revised unknown universe. By implementing the sampling plan, we will be able to determine the incidence of establishments and their corresponding employment of energy firms within each stratum of the unknown universe. Proper implementation of the sampling plan for the unknown universe will limit the need for adjusting weights for respondents to those stratum where firms identify they employ energy workers but are unwilling to complete the survey or do not finish the survey and their responses are not able to be used.

Step 3: Determining incidence and employment within each stratum and produce adjusted weights

In each industry stratum, data from the survey dispositions will provide the percentage of establishments and employment within each strata for energy workers and establishments from those that were sampled. For each stratum that is sampled in the unknown universe, the ratio between total employment for the sample and the revised unknown universe should be determined. The weighting adjustment only occurs within each stratum if the sample that is used for determining the disposition of energy establishments and employments differs from the completed surveys

Each stratum’s total revised unknown universe employment = ∑ UKEN _ILS

Each stratum’s total unknown sample employment⁶ = ∑ UKen _ILS

Each stratum’s total unknown employment from completed surveys = ∑ UKens _ILS

Weight for each stratum within the unknown sample only occurs when;

(∑ UKen _ILS ) / (∑ UKEN _ILS ) ≠ (∑ UKens _ILS ) / (∑ UKEN _ILS )

Weight for each stratum within the unknown sample = (∑ UKen _ILS ) / (∑ UKens _ILS )

B.4. Test Procedures and Form Consultations

Describe any tests of procedures or methods to be undertaken.

The data collection forms being submitted for approval have been field tested and received the highest rating of satisfaction of the forms being tested. Cognitive assessments will be conducted by EIA for refinement to the survey. Similar instruments have been used with success (with and without federal support) at least 30 times over the past 10 years

B.5. Statistical Consultations

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s) or other person(s) who will actually collect and/or analyze the information for the agency.

Josh Williams, President, BW Research Partnership, 760-712-6592

Philip Jordan, Vice President, BW Research Partnership, 857-919-0774

Ryan Young, Data Manager, BW Research Partnership, 831-601-5639

Mitchell Schirch, Research Director, BW Research Partnership, 603-204-4331

David Hiles (consulted previously), Supervisory Economist, Bureau of Labor Statistics, 202-691-6567

1 To remain in the known universe file, a firm needs to be reverse matched to DataAxleUSA’s national database or have some source of industry description. Employment by location is valuable but is not required to remain in the known universe.

2 Location is determined to be the entire state, for smaller states, or regions within a state defined by zip code for larger states.

3 In some cases employment by firm, among multiple locations is the only information that is available and in those cases employment is split equally among the locations it represents to provide an estimate of employment at each location.

4 Industry Employment Estimate is derived from QCEW

5 Industry Establishment Estimate is derived from CENSTATS

6 Total Unknown Sample employment is determined from survey dispositions not a complete survey