Protocol

Protocol for ACES Robotics Draft Question

ACES Robotics Question

Early Stage Scoping / Cognitive Interviews

Kristin Stettler

Draft 4.19.18

Research Questions:

• Do respondents understand what information they are being asked to provide with this question?

• Do respondents understand the terminology used in the question?

• Are respondents able to answer the question that is being asked?

• Is the requested information available in respondent’s records?

• Are the instructions helpful to respondents?

Expected Length of Interview: 30-60 minutes

Materials Needed:

• Draft question and ACES form

• Consent forms

• Digital recorder

General probes that may be used throughout the interview:

• Reflect back on respondent’s answer: “you said…”

• In your own words, what is this question asking?

• How did you arrive at this number/answer this question?

• What records (if any) did you look at? What line or lines were of interest?

• Specifically, what did you include in this number? What did you exclude?

• Would you consult other people to obtain this answer?

INTRODUCTION

• Explain purpose of meeting: get feedback on a potential new question for the Annual Capital Expenditures Survey on a new topic.

• There may be times throughout this interview that this seems like a test of you, I assure you that it is not. The purpose of these interviews is to ensure that we are communicating the data requests clearly with you, so in that way, these interviews are actually a test of us. On that note, we want your honest feedback throughout, and if it’s not clear to you how to answer, please do your best. There will be ample opportunities for you to let us know how to make improvements so that you could do a better job answering.

• As we go through the interview, we will ask both general and specific questions. If at any time a question seems odd to you, please let us know. We encourage all feedback.

• Before we start, I have a consent form that goes over the authority that we have to conduct these interviews. There’s also a piece in here where we ask for permission to record this interview, which is strictly for our note taking purposes. These interviews will only be heard by people directly involved in the development of the survey. Do we have permission to record our conversation for research purposes?

[Have respondent sign consent form. If they decline recording, note on the consent form that they declined recording and have them sign.]

ABOUT THE RESPONDENT

First, I’d like to learn a bit about you, and your role here at the company.

• What is your job title?

• What is your role in completing government surveys?

• How much experience do you have completing the Annual Capital Expenditures Survey?

• What has your experience completing the Annual Capital Expenditures Survey been like?

• Do you work with anyone else in your company to get the data for the Annual Capital Expenditures Survey?

  • How are those other people involved?

  • How many people are involved?

Now I’d like to learn a little about your company.

• Can you give us a brief description of what your company does?

• What kind of products do you sell/manufacture or services do you provide?

• [If manufacturing] Do you have any domestic or international manufacturing locations? How many?

• What sort of manufacturing or other equipment do you have?

• How is your company organized, in terms of locations, ownership, etc.?

SCOPING QUESTIONS

Thank you very much for that background information! Now, I’d like to transition to the main topic of today’s interview.

Terminology

I’d like to introduce you to some terms. For each of these terms, please define them in your own words, within the context of your company. It may also be helpful for you to provide us with examples, if any come to mind.

Interviewer:

• Allow respondent to define each term individually before moving on to the next.

1. Robots

2. Robotic equipment

3. Industrial robots

4. Service robots

• In your mind, are there major differences between these terms? Can you explain them to us?

• Are there any other terms that you may use that you consider synonyms of robots or robotic equipment?

• Can you give me some examples of industrial robots?

• Can you give me some examples of service robots?

Record Keeping

• In your financial records, are you able to identify:

  • Costs and values of robots/robotic equipment from the costs and values of other equipment?

  • Any other information about robots/robotic equipment?

• Does your company keep these records on robots/robotics at the company level? The establishment level? Some other level?

• If you are not able to access these records, does someone else have access to them, or are they simply not kept?

• Would someone, such as a plant manager, have access to other information or records that could be used to answer questions about robotic equipment?

  • What types of records? Financial, maintenance, etc.?

In Practice

• Are robots expensed differently than other assets like software or other related expenses?

• Are they acquired or leased?

• What other costs are associated with robots? (e.g., training, installation, maintenance, programming, safety precautions, etc.)

• Are you able to separate out these costs?

• Are you able to break down the costs between the robot, its software, and any other installation costs?

• Are robots/robotic equipment capitalized or expensed?

• Are the robots themselves expensed differently than their related software or other expenses?

• Who do you purchase the robots/robotic equipment from?

  • If needed: A robot manufacturer, a system integrator, both?

• Who services the robots/robotic equipment?

  • If needed: The manufacturer, an integrator, in-house, something else?

• Do the robots/robotic equipment physically interact with humans in a shared workspace, or are they fully autonomous?

COGNITIVE INTERVIEW QUESTIONS

Now I’d like to introduce you to the question that we have drafted.

First, I’d like you to review this page.

Interviewer: Allow them to read the instructions if they choose, but do not explicitly direct them to do so.

• In your own words, what is this question asking you?

• Do you have this information in your records?

• Do you have a ballpark figure for what this amount might be?

• What are you including in “capital expenditures”?

  • Is there anything discretionary about what’s included in capital expenditures, or does this follow strict accounting guidelines?

• If this question were included in the actual survey, would you be able to retrieve the data that’s being asked for, at the company level?

  • Is there anything difficult about retrieving this data?

• If you didn’t have anything to report in this question, would you enter a zero, check the Check if None box, or would you leave it blank?

• How long would it take you to answer this question?

• Did you review the introductory text before answering the question?

• Does this match the definition you provided earlier? If not, how is it different?

  • Does your company have any machinery that you would not personally consider to be a “robot” or “robotic equipment,” but you would be inclined to include because of the definition? What are they?

  • Are there any types of “robots” or “robotic equipment” that your company has, according to how you defined it earlier, that you believe this definition is telling you to exclude? What are they?

• What do you think is intended when the survey describes “industrial robots”?

  • What types of equipment would you include?

  • Exclude?

• What do you think is intended when the survey describes “service robots”?

  • What types of equipment would you include?

  • Exclude?

• [Review each line in the question/introductory text with respondent.]

• Would you like any additional clarifications/definitions in the introductory text?

• Would it be possible for you to report a percentage of total capital expenditures for robotics equipment?

• Would that be easier or more difficult than reporting a dollar amount?

• Would you prefer to be asked to report a dollar amount, a percentage or the option of either?

• Did you notice the note that “Estimates were acceptable”?

• Do you think it is helpful, or do you think it is unnecessary?

• Would you be comfortable providing an estimate?

• [Show them the current ACES form.] In which category in Item 2 of the current Annual Capital Expenditures Survey are you currently including robotics equipment?

Wrap-up

We are almost done, and I just have a few wrap-up questions.

• Do you think this question would be easy to answer, difficult to answer, or somewhere in between? Why?

• Is there anything else we haven’t touched on today, that you think is relevant?

• Do you have any questions for us?

Thank you very much for your time today!