OMB Memo

Request: The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). We will be conducting cognitive interviews for a series of proposed questions to be added to the Annual Survey of Manufactures (ASM).

Purpose: The Annual Survey of Manufactures is an annual survey, with coverage of manufacturing establishments, defined as establishments with one or more paid employees from the North American Industry Classification System (NAICS) sectors in 31, 32, and 33. It is a sample survey, with approximately 50,000 establishments included in the survey sample (approximately 15,000 of which are included with certainty). The purpose of the survey is to provide statistics on the manufacturing industry, on topics such as employment, payroll, worker hours, payroll supplements, cost of materials, operating expenses, value added by manufacturing, capital expenditures, inventories, energy consumption, and value of shipments. The survey is conducted annually, except for years ending in 2 and 7, when the data items become a part of the Economic Census. Data users include academics, economists, and policymakers. More information about the ASM can be found at https://www.census.gov/programs-surveys/asm.html.

The proposed questions are attached. These questions are two variations touching on the same concept, which is to measure the value of, and expenditures on, robotic equipment in the reporting year. The first variation of the question seeks to get information on the value of industrial robotic equipment at the end of the calendar year, and the capital expenditures on said equipment during that year. The second variation asks for the number of industrial robots used at the plant in 2018, and the average purchase prices of those robots, as well as the number of industrial robots purchased for the plant in 2018, and their average purchase price. In addition to these questions, the cognitive interviews will also test variations of definitions and include/exclude instructions. These questions and definitions are shown in Appendices A through D.

The goal of adding these questions to the ASM is to collect information on the use of robotics in manufacturing. Robotics have been increasingly used in manufacturing, but no Census Bureau economic surveys currently collect data on their use. Areas of research using this data may include the types of firms that use robotics, their relationship with human labor and employment rates, and how various aspects of the market might influence the use of robotics.

Staff from the Data Collection Methodology & Research Branch within the Economic Statistical Methods Division (ESMD) of the Census Bureau will be conducting cognitive interviews regarding this testing, with support from staff from the Center for Administrative Records Research and Applications (CARRA), the Office of the Associate Director for Economic Programs (ADEP), the Economy-wide Statistics Division (EWD), and the Manufacturing Surveys Statistical Methods Branch (MSSMB, within ESMD). For this testing, we will interview up to 40 respondents in two separate rounds. The 40 respondents will be randomly assigned to see one of the four variations of the test questions, which are Appendices A though D. This is done to control for ordering effects as it relates to a preference of the questions and the definitions. Appendices E and F are the two protocols that will be used to cover the variations in the survey questions; Appendix E covers versions A and B, while Appendix F covers versions C and D.

Method:

The method of research will be cognitive interviews, which are interviews aimed at understanding the way that a person interprets the items that are being presented to them, in order to understand and reduce potential measurement error before these questions are included in the survey. We will also be asking early stage scoping questions, which have a broader focus on the respondents’ understanding and familiarity with the concepts being addressed in the survey, the way their company’s records are organized, and the terminology that they use in reference to these concepts. These interviews will take place in-person at participants’ places of business or another location of their choosing. The interviews will follow a semi-structured interview protocol (attached).

Subject area specialists from the Census Bureau will participate in most, if not all, of the cognitive interviews. Their primary purpose will be for observation, but they may also assist in cases where additional clarification of the subject matter is required, and to provide information on the purpose and uses of the survey. Cognitive interviews will be audio recorded with the participants’ permission, to aid researchers in accurately summarizing key findings from the interviews.

Sample Population:

Staff from CARRA and ADEP will provide DCMRB staff with a list of businesses and respondent contact information, from which we will recruit. The frame of respondents will be recent respondents to the Annual Survey of Manufactures, with a particular focus on respondents who are expected to have an understanding of robotics.

Approximately 40 interviews will be conducted in total, with approximately 20 interviews in each of the two rounds. This number of interviews was selected because it is a manageable number of interviews for the time period allotted, and should be large enough to provide reactions to the questions and definitions that are representative of the survey population.

Forty interviews will allow us to cover four different cities in a cost-effective manner for testing, which we intend to do in order to meet with participants from companies covering a variety of industries. The specific sites of testing have yet to be determined.

Sample recruitment:

We will contact potential participants via phone, explain the nature of our research, and ask them to participate in our study. The sample of participants will be those who are able to be contacted and who agree to participate in the study. Efforts will be made to sample respondents with a variety of products that they manufacture, from variably-sized companies. During recruitment, we will screen respondents to determine if their company has any robotics equipment; if they do not, they will not be scheduled for an interview. The script for recruiting and screening participants can be found in Appendix G.

Participants will be informed that their response is voluntary and that the information they provide is confidential and will be seen only by Census Bureau employees involved in the research project. We will not be providing monetary incentives to participants in this study. Once interviews are scheduled, researchers will send respondents a confirmation via email, and may conduct reminder calls a few days before the meetings.

Timeline:

Recruiting for these interviews will begin as early as October 2017, for interviews that will be conducted during October 2017 through January 2018.

We anticipate that each interview will include one or two participants, for an average of 1.5 participants. We estimate that, on average, the interviews will take one hour. Thus, the total estimated burden for completed interviews is 60 hours (40 interviews X 1.5 participants X 1 hour).

Our goal is to schedule up 40 interviews. Potential participants who are interested will need to be screened to determine whether they have the types of machinery that could be considered robotics; we anticipate that to get 40 interviews, we may need to conduct 80 screening calls, which may take up to 10 minutes each. We anticipate that to reach 80 potentially interested persons, we may need to conduct 320 total calls; therefore, we anticipate up to 240 refusals or non-responses. The anticipated maximum burden for a refusal or a non-response is two minutes. As such, we estimate the maximum associated recruiting burden to be approximately 21.33 hours (80 screening calls x 10 minutes = 800 minutes; 240 refusals/nonresponses x 2 minutes = 480 minutes; 800 + 480 minutes = 21.33 hours). Thus, the estimated total public reporting burden for this research is approximately 81.33 hours.

Language:

Testing will be conducted in English only.

Enclosed are the draft questions, and the protocol that we will be using for the cognitive interviews.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Kenneth Herrell

Data Collection Methodology & Research Branch

Economic Statistical Methods Division (ESMD)

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-2197

[email protected]

cc:
Ron Jarmin          (DIR) with enclosures

Nick Orsini (DIR)        “          ”
Carol Caldwell             (ESMD)    “          ”
Diane Willimack             (ESMD)     “          ”
Amy Anderson Riemer  (ESMD)     “          ”

Carma Hogue (ESMD) “ ”

Kristin Stettler (ESMD) “ “

Danielle Norman (PCO) “ ”

Mary Lenaiyasa (PCO) “ “

Jennifer Hunter Childs (ADRM) “ “

Jasmine Luck (ADRM) ” “

Catherine Buffington (ADEP) “ “

Javier Miranda (CARRA) “ “

Edward Watkins III (EWD) “ “

Julius Smith, Jr. (EWD) “ “