Protocol

Services Annual Survey (SAS) E-Commerce Questions Cognitive Interview

DRAFT PROTOCOL

Kristin Stettler

January 31, 2017

This protocol is a guide – the questions presented here will not necessarily be asked exactly as worded in the protocol or in this order. It is important to note that not all questions will be asked in every interview.

Note: As much information about interview participants and enterprise/establishments should be obtained prior to the interview as possible. Conduct background research on each enterprise/establishment prior to each interview. In some cases probing may need to be adjusted based on the background research that was conducted.

Part A – Introduction (5 minutes)

1. Introduce observers and their background

2. Purpose of visit:

Thank you very much for your time. Let me start by telling you a little about what we will be doing today. We are working on some possible revisions to the Service Annual Survey, which is conducted annually and collects information on xx xx xx. We are in the process of redesigning portions of this form in order to better meet our data users’ needs. We would like to get your input on the revised questions, in order to make them as clear and easy to complete as possible.

3. Permission to audio record discussion? Have R sign consent form.

Before we get started, I would like to audio record this interview so I don't have to rely on my memory later. This session is confidential. Only persons connected with this project will have access to your recording. If that is all right with you, please sign this consent form. It also tells you about the confidentiality of this session.

Part B – Background Information (5 minutes)

1. Respondent Background

Before we start with more specific questions, I would like to ask you a few questions about this company and your role in it.

1. (For each person in the room) What is your formal job title?

1. Could you tell me a little about your role and responsibilities?

2. Have you completed the Services Annual Survey in the past?

3. How about other government surveys?

2. Business Background

1. What would you consider to be the main activity of this business?

Part C – Testing of Draft Questions (25-30 minutes)

Now, I wanted to review a draft copy of the Services Annual Survey. Here is a copy of last year’s form, if you’d like to glance through it to orient yourself.

Note: Probe retrospectively, after the respondent has completed all the questions.

If a respondent answers, “I don’t know” to any of these questions, follow up with these probes:

• Who in your organization would know about that?

• What is their title?

• Do they work at this location?

• (If no) Where are they located?

  • Continue to probe on any answers that seem to contrast previously reported answers.

OK, at this time, I will be sharing the current draft of the relevant questions with you. This is a draft of questions that would replace Question 8 in the current form I just showed you. The final version of these questions will eventually be part of an electronic version of the survey.

Question 8 – E-Commerce

A.  Did this firm record any operating revenue from sales generated through orders entered directly on the firm’s websites or mobile applications?

Yes / No (Go to C)

• How did you go about answering this question?

• Could you tell me what this question is asking for in your own words?

• What does “entered directly on the firm’s websites or mobile applications” mean to you?

• Can you give me some examples of the websites or mobile applications you were thinking of?

• Would you consult any records or anyone else in your organization in order to answer this question?

• Does this wording match how it is recorded in your records?

• How long would you estimate it would take to answer this question?

• Do you have any suggestions to improve this question?

B.  What was the percentage of total operating revenue recorded in Section 6 that was generated through orders entered directly on the firm’s websites or mobile applications?  Please provide an estimate if exact figures are not available.

_____ %

• How did you go about answering this question?

• Could you tell me what this question is asking for in your own words?

• Would you consult any records or anyone else in your organization in order to answer this question?

• Would you prefer to report this as a percentage of total operating revenue, or as an actual dollar amount?

• How long would you estimate it would take to answer this question?

• Do you have any suggestions to improve this question?

C.  Did this firm record any revenue from sales generated through orders entered through third-party websites or mobile applications, EDI systems, e-mail, or other electronic networks?

Yes/ No (Go to Section 9)

• How did you go about answering this question?

• Could you tell me what this question is asking for in your own words?

• What does “entered through third-party websites or mobile applications, EDI systems, e-mail or other electronic networks” mean to you?

• Can you give me some examples of the applications or systems you were thinking of?

• Would you consult any records or anyone else in your organization in order to answer this question?

• Does this wording match how it is recorded in your records?

• How long would you estimate it would take to answer this question?

• Do you have any suggestions to improve this question?

D.  What was the percentage of total operating revenue recorded in Section 6 that was generated through orders entered through third-party websites or mobile applications, EDI systems, e-mail, or other electronic networks?  Please provide an estimate if exact figures are not available.

_____ %

• How did you go about answering this question?

• Could you tell me what this question is asking for in your own words?

• Did you include ordered entered through e-mail?

• How significant was the e-mail portion?

• Would it be easier to provide an estimate if e-mail was not included?

• Would you consult any records or anyone else in your organization in order to answer this question?

• Would you prefer to report this as a percentage of total operating revenue, or as an actual dollar amount?

• How long would you estimate it would take to answer this question?

• Do you have any suggestions to improve this question?

Part D – Wrap Up (5-10 minutes)

1. Overall, what do you think of the questions we discussed today?

2. In general, what does the phrase “e-commerce” mean to you? Is it relevant to your business? Is it something you track?

3. What sorts of records does your business keep related to e-commerce?

4. Are there other words or terms you use to describe this concept?

5. Would you say that someone in your role would be an appropriate person to answer questions like these? If not, who would be the best person to answer questions like these?

6. How long would you estimate it would take to complete the questions we discussed today?

7. Do you have any other comments, questions or suggestions for us?

Thank you for your help today!

5