OMB Memo

Request: The Census Bureau plans to conduct new research under the Generic Clearance for Internet Nonprobability Panel Pretesting (OMB number 0607-0725). We plan to conduct a test of the questionnaire design for collecting information on goods and services, as defined by the North American Product Classification System (NAPCS), in the 2017 Economic Census.

Purpose: As part of the economic census, businesses provide detailed information about the revenue their business generated through the goods and services they provide. In past censuses, this reporting has been geared toward the products and services within the businesses’ main industry, under the North American Industry Classification System (NAICS). This classification system focused on the supply-side of the economy, providing a summary of where goods are produced. Beginning with the 2017 Economic Census, the Census Bureau will introduce a new way of collecting and disseminating the revenue information through the NAPCS framework. The focus of this framework is on the demand-side economy, providing a summary of where goods and services are sold, not just in the primary industry in which they are produced. (For additional information about NAPCS, see http://www.census.gov/eos/www/napcs/)

Under the NAPCS framework, businesses will be asked explicitly to report detailed information on all of the goods and services they offer that generate revenue, even if those goods and services are not typical of that business’ industry. With the NAPCS structure, although these goods and services might make up a fraction of the business’ revenue, respondents should still report these products, providing a description of them and their associated revenue.

The proposed design for collecting NAPCS products is to present respondents with a list of products typical to their industry. After selecting products, respondents will then be asked to report revenues associated with each of the selected products. The questionnaire includes additional open-ended response fields where the respondent can write-in other industry specific products (e.g., “other manufacturing revenue”) or any other good or service that generates revenue and was not pre-listed. Usability testing in 2015 indicated that this design worked well for the pre-listed products. However, the write-in fields for specific sectors within an industry did not perform well in testing.

Based on these results, future usability testing had been planned to evaluate alternative designs for labeling and organizing write-in fields. However, with the new NAPCS structure for publishing product revenue, write-ins play a key role in capturing the full range of business activities (products and services). While usability testing would be able to help determine a design for write-in fields, this method is limited in its ability to inform the quality of the information that respondents would provide and the magnitude of the impact, both in terms of reported receipts and the resources necessary to code and process NAPCS write-ins. Thus, the Census Bureau is undertaking this small-scale field test to evaluate NAPCS write-ins under real-world survey conditions. The results of this field test will help the Census Bureau identify and recommend a design for collecting NAPCS products for the 2017 Economic Census that maximizes data quality while reducing respondent burden.

 

Method

Design: We propose a split-ballot field test using the NAPCS product structure within an annual survey that collects product revenue details. Products would be presented in the NAPCS format that will be used for the 2017 Economic Census. Respondents would receive one of two possible presentation formats for including products not prelisted: Version A, “all other products” write-in fields only or Version B, the sector-specific write-in fields in combination with “all other products.” (See attached draft designs.)

Survey Population: The field test will take place using the 2016 Services Annual Survey (SAS), aka the Services Annual Report. This survey provides national estimates of revenue, sources of revenue, e-commerce revenue as well as total and detail expenses for service businesses in the US. For selected industries, only export revenue, end of year inventories and revenue by product and class of customers are collected. (See https://bhs.econ.census.gov/bhs/sas/about.html for more information on the survey.) Although SAS participation is mandatory, participants in this field test will be informed that their participation is voluntary in the survey invitation letter and on the survey’s welcome screen within the Web instrument.

Sample selection and Sample size: The Census Bureau identified six service industries for this field test. Table 1 lists these industries. These industries were selected because businesses within them are likely to sell products and provide services that do not represent major sources of revenue within their industry and would be unlikely to be pre-listed on the questionnaire (e.g., beauty shops collecting revenue from the sale of retail products). However, for the 2017 Economic Census, under the NAPCS structure, these businesses would be asked to provide revenue information on these out-of-sector goods and services.

Selected cases within these industries will include only single-unit businesses that had been in the 2015 SAS sample, but were not selected in the sample for the 2016 survey reference year. For 2016, SAS drew a new five-year sample from the Census Bureau’s Business Register. The cases used in this field test were not sampled in that procedure. Therefore, any data collected from these cases will not be used is producing estimates or in the processing of the 2016 survey year data. Table 1 shows a breakdown of cases by both industry and NAPCS write-in field version.

Table 1: Sample size by industry and Write-in Field Version
Industry	Write-in Field Version
	A	B	Total
811111 (General automotive repair)	68	69	137
811112 (Automotive exhaust system repair)	36	36	72
811113 (Automotive transmission repair)	37	37	74
812111 (Barber shops)	45	45	90
812112 (Beauty shops)	68	68	136
812113 (Nail Salons)	181	182	363
Total	435	437	872

Data Collection and Timeline: Although these cases will not be part of the production sample, data collection for this field test will parallel the cycle for the 2016 SAS. The current production cycle has the following milestones:

• Initial mail out - January 19th

• Due date reminder mailout - February 16th

• Due date - February 28

• Non-response follow up letter - March 14th

Results, Analysis, and Evaluation:

Data Analysis: Staff from the Economic Statistical Methods Division (ESMD) will analyze the write-in data collected from this field test to answer the following research questions:

• Are there differences in the number of write-ins that respondents provide based on the presentation of the write-ins?

• Are there differences in the specificity and the ability to code the write-ins that respondents provide?

• If there are differences, how would these they affect data quality?

Respondent Debriefings: From February through March 2017, as part of the evaluation, ESMD will conduct up to thirty respondent debriefing (fifteen per each version of the question) with respondents. The goal of these debriefings will be to garner feedback on any problems with reporting products using the write-in fields that respondents may have had.

Researchers from the Data Collection Methodology and Research Branch (DCMRB) will conduct these telephone interviews with respondents at times that are convenient to them.

DCMRB staff will recruit companies that participated in the field test, as soon after submission as possible. The interviews will either be conducted upon recruiting them or scheduled for a time that is more convenient for the respondent. If the respondent would prefer to schedule the interview for a later time, we will give participants a follow-up reminder of their appointments by fax or email. We will inform participants that their response is voluntary, the information they provide is confidential, and that only employees involved in the research project will see that information. We will not be providing monetary incentives to participants in this study.

We estimate that it will take respondents approximately 3 hours to complete the survey. The length of the interviews is expected to average 15 minutes. We expect to make up to five recruiting calls for every interview conducted, and each recruiting call will last an average of five minutes. Assuming 100% response, the maximum estimated respondent burden for this study is approximately 2,636 hours ((872 respondents survey X 3 hours) + (30 companies X 15 minutes) + (150 companies X 5 minutes)).

Attached are screen shots of the NAPCS write-in versions and the debriefing questions. The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Diane Willimack

Economic Statistical Methods Division

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-3538

[email protected]

Enclosures

cc:

Ron Jarmin (ADEP) with enclosures

Nick Orsini (ADEP)” “

Shelley Karlsson (EMD) “ “

Kim Moore (EWD) “ “

Andrew Baer (EWD) “ “

Katrina Washington (ESMD) “ “

Carol Caldwell (ESMD) “ ”

Carma Hogue (ESMD) “ “

Diane Willimack (ESMD) “ “

Amy Anderson Riemer (ESMD) “ “

Jennifer Hunter Childs (ADRM)” “

Jasmine Luck (ADRM)” “

Danielle Norman (PCO)“ “

Mary Lenaiyasa (PCO)“ “

4