Form 1 Debriefing Protocol

ACES Robotics Question

Respondent Debriefings

Kristin Stettler

Draft 12.17.19

General Research Questions:

• Do respondents understand what information they were asked to provide with this question?

• Do respondents understand the terminology used in the question?

• Are respondents able to answer the question that is being asked?

• Is the requested information available in respondent’s records?

• Are the instructions helpful to respondents?

Expected Length of Interview: 20-30 minutes

Materials Needed:

• Copy of questionnaire

• Information on Respondent’s answer (these can be screen shots or some other form of record of the participants’ responses)

General probes that may be used throughout the interview:

• Reflect back on respondent’s answer: “you said…”

• In your own words, what is this question asking?

• How did you arrive at this number/answer this question?

• What records (if any) did you look at? What line or lines were of interest?

• Specifically, what did you include in this number? What did you exclude?

• Did you consult other people to obtain this answer?

INTRODUCTION

• Explain purpose of meeting: get feedback on a new question that was added to the Annual Capital Expenditures Survey on a new topic.

• Thank respondent for completing the survey

• Tell respondent why we asked to speak with them

• Understand how they developed their answer

• Refine question and make it easier to answer

ABOUT THE RESPONDENT

First, I would like to learn a bit about you, and your role here at the company.

• What is your job title?

• What is your role in completing government surveys?

• How much experience do you have completing the Annual Capital Expenditures Survey?

• Do you work with anyone else in your company to get the data for the ACES?

  • How are those other people involved?

  • How many people are involved?

Now I’d like to learn a little about your company.

• Can you give us a brief description of what your company does?

SURVEY BACKGROUND

Thank you for that background information, it is very helpful. I would like to ask you some questions about your process for filling out the ACES.

• Can you tell me about your process for filling out the Annual Capital Expenditures Survey?

  • Additional probes (if necessary):

    • Was this your first time filling out this survey?

    • How do you typically gather the data?

    • Did someone else have to review the data before you submitted it?

    • How long did it take you to complete the survey?

SURVEY QUESTIONS

Now I would like to ask you about your responses to one specific question. Please open the attachment that I sent you via email.

Item 5b Robotic Equipment

On page one of the PDF, I would like you to look at item 5B, Robotic Equipment.

• Does your business have a definition of robotic equipment that it uses? (If yes) Please describe that definition. (If no) How would you define robotic equipment?

• How did you go about answering this question?

• Where did you get the data in order to answer this question?

• Did you need to consult with any records or anyone else while answering this question?

• (If the respondent reported a dollar amount): I see that you provided $ ___ dollar amount for this response. Can you provide me with examples of the specific robotic equipment you were thinking of in your response?

  • What types of equipment were you including?

  • Excluding?

• Did you include new and used robotic equipment?

• Did you include software, installation and other one-time costs?

• Was the value reported for software only?

• Any other comments on what you included or excluded in your answer?

• Did you review the introductory text before answering the question?

• What is your reaction to the definition on the left side?

• Does this match the definition you provided earlier? If not, how is it different?

  • Does your company have any machinery that you would not personally consider to be a “robot” or “robotic equipment,” but you included because of the definition? What?

  • Are there any types of “robots” or “robotic equipment” that your company has, according to how you defined it earlier, that you believe this definition told you to exclude? What?

• Do you have any suggestions for improving the definition?

• Looking at the list of tasks (on the right) performed by robots, what is your reaction? Are there any that surprised you? Any that you feel are missing?

• Looking at the list of Excludes, what is your reaction? Are there any that surprised you? Any that you feel are missing?

• Did you enter anything in the Remarks below the question?

• How long did it take for you to complete just this question? How about the entire Annual Capital Expenditures Survey?

WRAP-UP

We are almost done, and I just have a few wrap-up questions

• This question asked only about capitalized expenditures. Do you have any robotic equipment that was not capitalized? How do you determine when equipment is capitalized or expensed? What is the capitalization threshold and does it vary by type (e.g., maintenance/repairs v. new equipment)?

• Is there anything else we haven’t touched on today, that you think is relevant?

• Do you have any other recommendations for improving the survey that we didn’t get a chance to talk about?

• Do you have any questions for us?

Thank you very much for your time today!

6