Department of Commerce
U.S. Census Bureau
OMB Information Collection Request
Manufacturers’ Unfilled Orders Survey (MA-3000)
OMB Control No. 0607-0561
Part B -- Collection of Information Employing Statistical Methods
1. Universe and Respondent Selection
The target population for the Manufacturers’ Unfilled Orders (M3UFO) Survey is defined as all domestic companies with five or more employees that have one or more manufacturing establishments classified in any of the 2012 North American Industry Classification System (NAICS) industries that make up the 42 M3 industry categories believed to maintain unfilled orders. Eight of these industry categories are each broken into two categories, a defense component and a nondefense component. Therefore, the M3UFO survey actually covers 50 M3 industry categories. These 50 industry categories encompass the following major groups in manufacturing:
Primary Metals
Fabricated Metal Products
Machinery
Computers and Electronic Products
Electrical Equipment, Appliances, and Components
Transportation Equipment
Furniture and Related Products
Miscellaneous Products
The industries selected for the M3UFO survey are those which the U.S. Census Bureau determined to maintain considerable unfilled orders. The M3UFO survey is necessary to ensure future accuracy of the unfilled orders and new orders data in the Manufacturers’ Shipments, Inventories, and Orders (M3) Survey and to determine which NAICS industries continue to maintain unfilled orders. There is a clear need for periodic benchmarking of the M3 survey estimates to accurately reflect the manufacturing universe, and the M3UFO survey provides these benchmarks for unfilled orders data.
2. Procedures for Collecting Information
The sampling frame for the M3UFO survey is constructed using information extracted from the Census Bureau’s Business Register and updated with measure-of-size (shipments) data from the Economic Census. This sample frame represents roughly 72,000 manufacturing companies located in the United States having five or more employees and with establishments classified in any of the 50 M3 industry categories described above.
The initial sampling frame for the M3UFO survey is comprised of company by M3 industry category records. Companies with activity in multiple industry categories have multiple records in this initial frame. Each company by industry category record represents the combined activity of all establishments classified within that industry category.
The sample for the M3UFO survey is selected using a probability-proportional-to-size (pps) sample design based on the assigned measure of size. Sampling is controlled at the M3 industry category level and each company is assigned a probability of selection based on its respective measure of size for each industry in which it has activity. Therefore, each company by industry category record in the initial frame is assigned a probability of selection that is commensurate with its relative importance (based on value of shipments) within the respective industry category. Consequently, a company with activity in multiple industry categories is assigned multiple probabilities of selection. The final probability of selection assigned for each company is the maximum of these industry-based selection probabilities. We also impose a minimum probability rule, which results in a maximum sample weight (inverse of the minimum probability). This minimum probability rule reduces the risk of exceedingly large sample weights that can adversely impact sample estimates and variances. For the M3UFO survey, the minimum probability is 0.01, resulting in a maximum sample weight of 100. There are also some companies specified by the M3UFO survey analysts to be included in the sample as arbitrary certainties. These companies are flagged up front and included in the sample with weights of 1. The sample is allocated across the 42 unique M3 industry categories that are in scope of the M3UFO survey based on industry category priorities. Ultimately, the sample selection process for the M3UFO survey results in a final sample of 6,000 companies.
Unfilled orders estimates are produced for each of the M3UFO industry categories using a ratio estimator. The industry categories that have both defense and nondefense components are split based on allocations obtained from actual M3 data. Horvitz-Thompson (HT) weighted estimates of unfilled orders and value of shipments are derived for each industry category using only companies in the M3UFO sample that have reported data for both data items, provided that shipments data are greater than zero. The sample weight assigned to each company is the inverse of its probability of selection. Ratio estimates of unfilled orders are produced for each industry category by constructing a ratio of the weighted estimate for unfilled orders to the weighted estimate for value of shipments (UO/VS) from the M3UFO survey and applying this ratio to the corresponding Annual Survey of Manufactures (ASM) or Census of Manufactures total value of shipments estimate. Therefore, the unfilled orders estimates for each industry category produced from the M3UFO survey are derived using the following formula:
[(HT estimate of unfilled orders) / (HT estimate of value of shipments)] * ASM or Census total value of shipments estimate
Similar to the M3 monthly link-relative estimation procedure, when a large company reports an unusual ratio of unfilled orders to shipments for a given industry category and strongly influences the overall industry ratio, the company’s data are excluded from the UO/VS ratio for that industry. However, the company’s unfilled orders data are included in the final unfilled orders estimate for the industry category.
Estimated coefficients of variation (CVs) are computed for all unfilled orders estimates, at the total manufacturing level and for all M3UFO industry categories. Sampling was controlled for each industry category under CV constraints ranging from 0.015 to 0.20, depending on industry priority. For the most recent M3UFO survey, actual estimated CVs ranged between 0.01 and 0.18 at the industry category level, and the estimated CV at the total manufacturing level was roughly 0.02, so all reliability requirements were satisfied.
3. Efforts to Maximize Response
We calculate two measures of response for the M3UFO survey. The check-in rate measures the unweighted proportion of questionnaires returned, with or without data, out of the total number of questionnaires mailed. The weighted response rate measures the weighted contribution in terms of measure of size (value of shipments) for active companies with reported data included in the final unfilled orders estimates relative to the total weighted measure of size for all active companies in the M3UFO survey. Based on previous years, we expect a check-in rate of at least 80 percent and a weighted response rate of at least 70 percent. For the most recent M3UFO survey, the check-in rate was approximately 81 percent and the weighted response rate was also roughly 81 percent.
Dedicated efforts are made to maximize the check-in and weighted response rates. For 2016, a reminder letter was added to the M3UFO collection strategy in an effort to reduce the number of respondents that are classified as past due as of the 30-day reporting deadline. The reminder letter was sent to non-respondents 20 days after the initial mailing of the form. Companies that did not respond to the survey after 30 days received an additional follow-up letter. Forty-five days after the follow-up letter, a telephone follow-up was conducted of non-respondents for a series of four call attempts over a four-week window.
In an attempt to boost the weighted response rate, a second telephone follow-up series was added for 2016 where the largest companies of each M3UFO industry category were contacted by phone seven days after the initial telephone follow-up period had been concluded. Each of these companies was contacted until a valid response was acquired by an M3UFO analyst. The additional of this second follow-up strategy has resulted in weighted coverage rates increasing from 70% to 81% since the last OMB reauthorization of the M3UFO survey.
The M3UFO survey will continue to use the last two years’ mailing strategy going forward by sending an initial form to respondents, followed by a reminder letter and follow-up letter to non-respondents. A phone strategy consisting of telephone follow-up for non-respondents with a subsequent round of telephone follow-up for the largest non-respondent companies will be used going forward for the purpose of achieving higher check-in and weighted coverage rates.
Report forms are reviewed and evaluated for any comments and suggestions made by the respondents to improve the survey results.
4. Tests of Procedures or Methods
Periodically, the Census Bureau staff interviews survey respondents to assess our data requests and to keep abreast of the current record keeping practices and changes in technology.
5. Contacts for Statistical Aspects and Data Collection
Person responsible for statistical methodology:
Amy M. Newman-Smith,
Methodology Director for
Manufacturing, Investment and Construction Programs
Economic Statistical Methods Division
U.S. Census Bureau
(301) 763-6595
Person responsible for data collection:
Mary Catherine Potter, Assistant Division Chief
Quarterly and Manufacturing Indicator Programs
Economic Indicators Division
U.S. Census Bureau
(301) 763-4207
List of Attachments
A. Form MA-3000
B. Instructions MA-3000(INST)
C. Major Manufacturing Activities List MA-3000(I)
D. Initial Mail-out Letter
E. Reminder Letter
F. Past Due Letter
G. Centurion Instrument Screenshots
| File Type | application/msword |
| Author | young006 |
| Last Modified By | SYSTEM |
| File Modified | 2018-07-12 |
| File Created | 2018-07-12 |