E-Commerce Exploratory Interviews for AIES

Attachment K

Department of Commerce

United States Census Bureau

OMB Information Collection Request

Annual Integrated Economic Survey

OMB Control Number 0607-1024

E-Commerce Exploratory Interviews for AIES

September 30, 2024

Cognitive Testing Support Project: Final Report on Exploratory Interviews of the e-Commerce economic activity for the Annual Integrated Economic Survey (AIES)

Final Report

Prepared for

Economy-Wide Statistics Division

Economic Statistics and Methodology Division

The U.S. Census Bureau

Prepared by

Y. Patrick Hsieh, PhD, Katherine Blackburn,

and Chris Ellis

RTI International

3040 E. Cornwallis Road

Research Triangle Park, NC 27709

The Census Bureau has reviewed this data product to ensure appropriate access, use, and disclosure avoidance protection of the confidential source data (Project No,-P-7530157, Disclosure Review Board (DRB) approval number: CBDRB- FY25-ESMD009-002).

Contents

Section Page

1. Introduction 1-1

2. Methodology 2-1

2.1 Goals And Research Questions 2-1

2.2 Research Design 2-1

2.2.1 Population of interest 2-1

2.2.2 Participant recruitment 2-1

2.2.3 Debriefing interview protocol and procedure 2-3

3. Findings from the Debriefing Interviews 3-1

3.1 Previous Research on e-Commerce Activities by Census Bureau 3-1

3.2 How Businesses Define e-Commerce Activities 3-2

3.2.1 e-Commerce activities involve a point of sale online 3-2

3.2.2 Difficulties in defining e-Commerce activities 3-3

3.2.3 Varying industry-specific considerations of e-Commerce activities 3-5

3.2.4 Scope impact on e-Commerce definition 3-9

3.2.5 Does the business conduct e-Commerce activities? 3-10

3.3 Review of the Current AIES e-Commerce Question 3-10

3.3.1 Reactions to question text without instruction 3-10

3.3.2 Reactions to question text with instructions 3-14

3.3.3 Scope impact on e-Commerce definition 3-27

3.3.4 Visibility of instructions 3-28

3.4 e-Commerce Record-Keeping Practices 3-29

3.4.1 Levels of e-Commerce records and tracking 3-29

3.4.2 Data availability impacts understanding of e-Commerce 3-39

3.4.3 Accessibility of e-Commerce records 3-41

3.5 Recommendations and Next Steps 3-44

3.5.1 Recommendations for the AIES question text 3-44

3.5.2 Recommendations for future research 3-47

4. Concluding Remarks 4-1

Appendix A. Exploratory Interview Protocol A-1

Appendix B. Current AIES e-Commerce Question B-1

Exhibits

2-1. Sample Characteristics and Recruitment Outcomes 2-2

3-1. Amount of e-Commerce Activities Reported by Participants 3-9

3-2. Current AIES e-Commerce Question Text 3-11

3-3. Current AIES e-Commerce Question Text Instructions 3-15

3-4. Types of Businesses Tracking e-Commerce Activities 3-30

3-5. Tyes of Businesses Tracking e-Commerce Activities by Non-profit versus For-profit Status 3-31

3-6. Accessibility of Records by Types of Businesses Tracking e-Commerce Activities 3-42

1. Introduction

The Annual Integrated Economic Survey (AIES) is a re-designed survey to integrate and replace seven existing annual business surveys into a single streamlined instrument. The goals of the AIES redesign are to provide an easier reporting process for businesses, collect better and more timely data, and reduce costs for the U.S. Census Bureau through more efficient data collection. Designed to be conducted annually, the AIES will provide key yearly measures of economic activity, including the only comprehensive national and subnational data on business revenues, expenses, and assets.¹

As part of the research effort for improving AIES, the Census Bureau contracted RTI International and Whirlwind Technologies, LLC, to conduct exploratory interviews with respondents who completed the AIES web instrument to survey and document the record-keeping practices and common verbiage for e-Commerce economic activity for future iterations of the AIES. This interviewing represents an additional research methodology to explore mis- and under-reported content on the AIES specific to e-Commerce. From the interview feedback, the project team was able to explore the record-keeping practices and data accessibility related to e-Commerce activities and how such practices reflected opportunities and challenges in measuring e-Commerce activities in the AIES context.

This report summarizes that effort, thematic research findings, and ensuing recommendations for future research.

2. Methodology

   1. Goals And Research Questions

This qualitative study explored the reporting experiences of AIES respondents, focusing on their definition of e-Commerce business activities, their record-keeping practices and data accessibility related to e-Commerce activities, and their reactions to the wording and definition of the e-Commerce question in the current AIES. The findings assess how record-keeping practices and data accessibility related to e-Commerce activities may reflect varying definitions of e-Commerce activities and subsequent considerations of the need to track such activities to support core business functions. The findings from the exploratory interviews will enable the Census Bureau to consider further research directions and potential designs for updated survey questions to measure e-Commerce activities. This study contributed to the continuous efforts by the Census Bureau to improve the AIES further.

2. Research Design

   1. Population of interest

The population of interest for debriefing interviews consisted of businesses that completed the AIES instrument after April 1, 2024. Because of the rapid turnaround needed to meet the debriefing interview schedule, only businesses for which the Census Bureau had an email address and phone number in the dataset were eligible for this study. Census Bureau staff from the Economy-Wide Statistics Division (EWD) provided the project team with a sample list of businesses that completed the AIES instrument on a rolling basis throughout the debriefing interview data collection period. This sample list was used to contact potential respondents.

In addition to contact information, the EWD staff were also able to append to the recruitment file several key business characteristics, including the total number of establishments, North American Industry Classification System (NAICS) code, whether the business had establishments in Puerto Rico, whether the business was in the manufacturing sector, and whether the business reported any e-Commerce revenue. However, these key characteristics were not available for all businesses and were used to track sample diversity of completed interviews rather than for targeted recruitment. The total number of establishments was used to categorize businesses into three different sizes: single unit, small business with less than 10 establishments, and large business with 10 establishments or more. Because of missing NAICS code data, business sector information was confirmed during the exploratory interview.

2. Participant recruitment

The goal of participant recruitment was to conduct approximately 30 total exploratory interviews. Consistent with the previous e-Commerce research effort by the Census Bureau, conducting 30 exploratory interviews from a diverse group of businesses was deemed adequate to provide sufficient feedback on the record-keeping practices and data accessibility e-Commerce activities. In April 2024, Census Bureau staff shared the first sample list with the project team to recruit participants for the debriefing interviews. As the project team proceeded to contact and solicit research participation from the sample list, the breakdowns of sectors and business size of completed interviews were also monitored. Only businesses that reported e-Commerce revenue in the 2023 AIES were sampled for participation in this research effort.

Data collection began in June 2024, and concluded in July 2024. The project team contacted 541 sampled businesses and completed 31 interviews; one did not attend their scheduled interview, four withdrew participation during the interview, and another 29 refused to participate. Among the 31 completed interviews, three participants represented a single unit company, 10 participants represented a small company with less than 10 establishments, and 18 represented a large company with more than 10 establishments. Three participants represented a manufacturing company, and another three participants represented a company with establishments located in Puerto Rico. Exhibit 2-1 details the distribution of completed interviews by size of business.

Exhibit 2-1. Sample Characteristics and Recruitment Outcomes

n	Interview status by type
	Number of units
	Single Unit	Small (<10)	Large (≥10)
Total sample	152	183	206
Completed	3	10	18
No show/withdrew/refused	6	12	16

Data collection began with the following contact protocol: one email followed by one call, then a second email for all sample members. Because of low engagement and the rapid timeline for data collection, this contact protocol was adjusted to focus primarily on emails, which allowed for a wider reach of potential participants more quickly. All 541 sample members received at least one email communication and 203 were contacted by phone to solicit participation. Following calls, 49 sampled members received either a second email invitation or a second phone call as the last recruitment attempt.

Participant recruitment was challenging due to several factors. Sample members refused to participate in the interview primarily because of the time commitment; many businesses were engaged in end of fiscal quarter activities, and typical points of contact for Census Bureau surveys were overwhelmed with other work. Future data collection efforts with AIES respondents should be sensitive to timing around year- or quarter- end activities and the deadline for tax filing and other regulatory compliance activities of businesses.

3. Debriefing interview protocol and procedure

Exploratory interviews, as a qualitative method, are typically employed at an early stage in the instrument design and evaluation process.² Exploratory interviews tend to use a more wholistic approach to ask participants to respond to the exploratory questions or probes in an open conversation. This method is suitable for collecting insights into a theoretical or empirical concept of focus for operationalization for developing survey questions.

Census Bureau staff from EWD provided the project team the following three main research questions for review:

1. What are the current e-Commerce record-keeping practices of businesses?

• What is included or excluded in these records?

• What are the differences in record-keeping by firm industry(ies)?

• What are the differences in record-keeping by firm size and complexity?

• At what level of granularity are these records kept?

2. How accessible are records of e-Commerce activity?

• How easy or challenging is accessing this information?

• How many people at the business are involved in maintenance and pulling of e-Commerce records for reporting?

• Are there levels of granularity that are more or less accessible for respondents (e.g., establishment level compared to industry or company-wide level)?

3. How do current e-Commerce record keeping practices inform current reporting practices on the AIES?

• What are respondents currently including (and excluding) from reporting e-Commerce for AIES?

• How much data manipulation (aggregation, allocation, estimation, and others) do respondents engage in to report e-Commerce for AIES?

• What are the factors that determine the decision to report e-Commerce on the AIES?

Based on these research questions, the interview protocol was further revised to enhance the clarity of the debriefing questions and the flow of the protocol.

During data collection, project team survey methodologists conducted all exploratory interviews virtually using Microsoft Teams. Participants were encouraged to join the interviews via the computer so that interviewers could share their screen to review the current AIES question. In a few cases when screen sharing was not possible, interviewers emailed a PDF document with the question text to participants.

Before the interview, participants were asked to review the consent document, which was sent via email. At the start of each interview, the main points of consent were reviewed, and the interviewer confirmed that the participant consented to be interviewed. Additionally, participants were asked to consent to the recording of the interview. If consent was provided, the interviewer began the recording. The interviewer then proceeded with the exploratory interview following a semi-structured interview protocol (Appendix A).

The protocol first included questions that gauged participants’ experience in completing the AIES, how participants retrieved information, participants’ perception of the difficulty of the questions, and what level of burden might be associated with data retrieval and reporting. The first section of the protocol set up an opportunity for interviewers to establish rapport with participants by discussing any problems they faced in responding to the newly formatted AIES. This approach relied on the social exchange of feedback about the AIES in exchange for an in-depth discussion about the e-Commerce activities of participants’ business. The protocol also included questions that asked about the definition of e-Commerce activities of the businesses, and their record-keeping practices and data accessibility for such activities. Lastly, the protocol also included questions that asked about participants’ reactions to the e-Commerce question in the current AIES. Interview length was approximately 45 to 60 minutes, and the study did not offer monetary incentives for research participation.

3. Findings from the Debriefing Interviews

This section begins by presenting the findings from Module 2 of the interview protocol (see Appendix A), in which participants were asked to define what e-Commerce means to them. Although the purpose of this research effort was not to focus on the definition of e-Commerce because previous Census Bureau research has covered this extensively, the participant-provided definitions are key to the topics of interest: record-keeping practices and data accessibility. After reviewing participant definitions of e-Commerce, the report will outline reactions to the current AIES e-Commerce Question (Module 5) and how the instructions provided do and do not fit with participants’ conceptualization of e-Commerce. The last two parts of this section will cover the key research topics of record-keeping practices and accessibility of e-Commerce records (Module 3 and 4, respectively).

1. Previous Research on e-Commerce Activities by Census Bureau

The topic of e-Commerce is one that the Census Bureau has been exploring since the beginning of electronically coordinated commerce. Previous research has shown that e-Commerce is challenging to measure and define for several reasons. Respondents have previously reported difficulties understanding the concept of e-Commerce and lack of access to e-Commerce data. As defined by Fayyaz (2018),³ e-Commerce consists of three aspects—the ordering and delivery process, the nature of the products, and the actors involved—thus leading to a three-prong definition of e-Commerce that consists of the following:

• Digitally ordered transactions (i.e., sales conducted over computer networks),

• Digitally facilitated transactions (i.e., cross-border trade flows facilitated by online platforms, such as credit card and bank systems), and

• Digitally delivered transactions (i.e., services delivered digitally as downloadable products³).

The exploratory interviews from which these findings are reported focus less on the established definitional issues and more on the issues of record-keeping and accessibility of said records. However, each participants’ definition is still key to understanding their reports about their records; participants only reported the records associated with e-Commerce as they understand the concept. Additionally, review of the current AIES e-Commerce question, which participants had responded to in the 2 months before interview, was useful to explore the definitional issues and further understand participants’ records. This research hopes to report an updated perspective on how businesses organize and access records about their e-Commerce activities.

2. How Businesses Define e-Commerce Activities

   1. e-Commerce activities involve a point of sale online

When participants conceptualize e-Commerce, one of the key features is that the point of sale (POS) is online. Other terms that consumers use in addition to “online” are “electronically” or “electronic,” “digital,” and “virtual.” For many businesses, this definition of e-Commerce includes that the POS is remote from the business and taking place through a website, either the business’ own website or a third-party app/website (e.g., DoorDash). This conceptualization maps well onto the first prong of Fayyaz’s (2018) definition that specifies e-Commerce as digitally ordered transactions; however, participants think more about the sale (i.e., when and how customers are paying) rather than placing orders or reservations. One archetype of e-Commerce that many participants think about is Amazon; this business features a primarily online storefront, no in-person interactions, and products delivered remotely to consumer’s homes.

“For me I would immediately think of companies like Amazon, PayPal, eBay, where you buy or sell something online.”

“I would define e-commerce as any transaction that is being entered electronically or online, via a website or an internet portal… I personally have probably like a lot of people, when I think of e-Commerce, the first thing I think of is buying something on Amazon.”

Participants also think about what e-Commerce is not when they define it. These definitions include not in person (e.g., “non-brick-and-mortar sales”) and not with physical cash. One participant specifically excluded bulk purchases to other businesses because they are not ordering through a customer-facing website. Others excluded purchases made by mail or phone and paid by check.

“I would say receiving premiums electronically, not physical cash. The majority of our premium revenue is through our online system. It is electronically transferred; we aren’t handling physical money.”

“But normally, our large customers are not going to order from Amazon, they want a large quantity. If they want to try our [product], then we could do a sample order, and then we would send them special packaged sample quantities that we could send to them for that. And that’s processed differently [than Amazon purchases].”

Participants also understand e-Commerce through what their business offers and how their business operates. They provided several examples of how the business they work for operate when defining e-Commerce, including options to buy online and pick up in store (BOPIS), electronically delivered goods and services (e.g., digital marketing, software, telehealth), and donations for non-profit organizations. Participants whose businesses have more diverse methods of e-Commerce (e.g., BOPIS and ship-to-consumer orders) seem to have a broader definition of e-Commerce, with more inclusive examples of what counts as e-Commerce. Other less common methods that participants considered e-Commerce included in-store purchases made on the business website for delivery to customers’ homes and orders placed via email.

2. Difficulties in defining e-Commerce activities

Participants’ definitions of e-Commerce align primarily with the first prong of Fayyaz’s (2018) definition, with a focus on orders placed and paid for online. However, participants struggle to conceptualize business practices that do not neatly fit into the archetype of online retailers like Amazon, especially online reservations for in-person services. Six participants were uncertain whether instances of online reservations for in-person services should be considered e-Commerce because the service itself is provided in person.

“I mean when you go to a Bruce Springsteen concert, I don’t think of that as e-Commerce. If you are purchasing access to a concert in an arena, even though your purchase was done electronically, that doesn’t feel like e-Commerce to me.”

“We have internet transactions, but there is ultimately a service/rental/product that is being delivered…We have half of the e-commerce transaction not the full e-Commerce transaction. We would generate a sale from the golf course. You can designate your tee time on the internet and pay—that would be e-Commerce but only partially because you have to come out and play golf. So, it would only be half the transaction. True e-Commerce…is you buy a product and I send it to you—for example, Amazon or any online retailer.”

“In our particular case it does not apply yet. I think of it as someone goes online and pays for goods or services. We currently don’t have that feature, someone can go online and make a reservation, they come in and pay directly through a face-to-face transaction.”

A complicating factor for some participants is when services are reserved online in advance but payment is only provided at the time the service is provided. When payment is provided in person, participants do not consider this e-Commerce. For example, one participant explained that currently their services are reserved through a website but payment is collected in person, which they would not consider e-Commerce. However, the business is working to store credit cards on file, which would be charged automatically, and the participant considered this as e-Commerce.

“In our particular case [e-Commerce] does not apply yet. I think of [e-Commerce] as someone goes online and pays for goods or services. We currently don’t have that feature, someone can go online and make a reservation, they come in and pay directly through a face-to-face transaction…When the card on file becomes active, I would count it as e-Commerce because it will be part of our compliance, that’s why it is clear because right now all of our transactions are face to face.”

Like with services delivered in person, there is some confusion about BOPIS for goods. Although BOPIS is clearly e-Commerce for some participants, others question whether receiving the product in person can be considered e-Commerce. Five participants centralized the delivery method as key to their definition of e-Commerce. The dominant image of e-Commerce as exemplified by stores like Amazon can sometimes obscure participants’ understanding of e-Commerce when goods are not delivered. One participant who prioritized delivery method in her definition of e-Commerce believed that only products provided via mail to a consumer’s home would count as e-Commerce.

“I think of [e-Commerce] as you place an order online or through the internet and they are mailed to you. I’m not sure what the official definition is. So, I may have answered that we didn’t do e-Commerce, because you have to pick [our product] up in person. I’m just not sure about that…I think our website you can place an order and pick it up. But we’re food, we don’t mail anything back. The only thing would be placing an order online, through UberEats or DoorDash, it’s not like Amazon where it can get shipped out. So, in that sense no [we don’t have e-Commerce].”

“To me [e-Commerce] is internet sales, buying off of the internet without going to a brick-and-mortar store.”

Delivery method can also be a point of confusion for electronically delivered services and products (e.g., software), which map directly to prong three of Fayyaz’s (2018) definition of e-Commerce. One participant who provides digital marketing was conflicted because the work (i.e., photographing, client meetings, editing) happens in person but the final product is delivered electronically:

“It’s confusing to me as we have things that are delivered electronically, but the work is done on-site… I don’t know if what I am doing is e-Commerce. The product might be electronic, but the service is reliant on in-person interactions.”

Several participants were concerned about the “moving line” of what could and could not be considered e-Commerce. As they talked more, they started to wonder whether all transactions could be considered e-Commerce if they were tracked in an electronic system or made with electronic forms of payment (e.g., credit cards, bank transfers).

“If I go to Whole Foods, and I load up my cart, and I pay for it at the cash register. That’s not e-Commerce, right? However, the way I pay at Amazon, is I hold my palm over the thing, and I never take out a credit card. It charges my card based on scanning my palm. That sounds very electronic to me. Where is the line?”

“…at what point would something not be considered e-commerce in our business environment today? Even if someone pays by cash, it’s still getting input into a software and it’s being reported to you that a sale took place. I would probably draw the line if you’re there, even if paying electronic means or apple pay on your phone, I would probably not consider that e-commerce.”

“I guess I’m being too granular about this, but you’re right when does eCommerce start or stop?”

3. Varying industry-specific considerations of e-Commerce activities

As outlined in Section 2, the sample of completed interviews included representation from diverse business types, each with their own understanding and application of the concept of e-Commerce. This section will highlight the key findings for several industries that may not fit the description of the e-Commerce archetype that many participants had considered, including non-profit organizations, utility companies, and companies with salespeople.

Non-profit organizations

As Census Bureau’s previous research has illustrated,⁴ non-profit organizations often have unique industry-specific comprehension challenges. When shown questions written with for-profit businesses in mind, non-profit organizations can struggle to understand business terminology (e.g., sales, total revenue, goods or services). Eight participants representing non-profit organizations participated in the current study. Most of these businesses included donations that were remitted electronically in their definitions of e-Commerce. They were more likely to mention electronic funds transfer (EFT) transfer, wire transfer, and automated clearing house (ACH) transfer than for-profit businesses. However, three participants did not initially consider grant funding and electronic donations as e-Commerce. One participant explained why she did not include grants as e-Commerce:

“We do very little e-Commerce because again we are 95% grant funded. Those come through grantor funding source, our services to individuals are free of charge. So that type of business is not what I would consider e-Commerce. I’m not going to the TikTok shop and buying something.”

Non-profit organizations do not fit well into the archetype of Amazon as e-Commerce because their revenue-generating activities may be entirely donation- or grant-based rather than focused on providing goods or services. Further discussion about non-profit organizations is included in Section 3.3.2 in review of the current AIES question.

Applicability for non-retail business models and business activities operated under contracts

There were also interesting e-Commerce applicability issues related to either industry-specific business models of operation or business activities operated under contracts experienced by non-retailers in the current study, such as utility companies, banks or financial services, and companies that lease out space or equipment. Two participants struggled to understand whether going online to pay bills should be considered an e-Commerce transaction. Another business reported that they leased out equipment and did not consider those transactions as e-Commerce, even when the bills were paid via electronic methods by their lessee. However, this company did consider “donations that are coming to us electronically” as e-Commerce activities.

“No, we don’t classify ourselves as an e-Commerce [company]. We do not offer a website to our customers to buy our products. Our marketing department has been handling the business sales of our products. They work out sales contracts with our customers. They will transmit the products to the destined locations for our customers to fulfill the transaction.”

“I think it’s the nature of our business model. It’s not like a person who lives in a residential home goes onto our website and orders in a quantity. We supply it. It’s different than Amazon or Target where you go online and you buy a widget. I think of e-Commerce in that way. Even Etsy—a person can sell individual things. [It’s] transactional. I don’t think of us using websites and customers coming on and buying gas like that or selling it or asking to store it. I mean we have a website but customers don’t go online and select a product and add it to their cart through the payment process in the traditional sense.”

“[Leasing] is not e-Commerce. Some people will choose to pay us as e-Commerce, but I don’t count those as part of our e-Commerce numbers because those companies are making a lease payment in whatever way is most convenient to them. The majority send us a paper check.”

Two participants reporting for banks were also included in this study and had drastically different interpretations of e-Commerce. One of these participants believed all of their revenue could be considered e-Commerce because all of the transactions take place digitally, including all fees earned for the accounts and interest. However, the other participant thought only a small part of their financial service operations could be considered e-Commerce, when the business conducts an internet auction for unsold off-lease vehicles. This participant did not consider the loans (i.e., the financial service) provided for the car dealerships as e-Commerce.

These examples illustrate the difficulties with understanding Fayyaz’s (2018) second definitional prong of digitally facilitated transactions. Even businesses that operate entirely digitally, such as a bank providing loans, may struggle to incorporate these digitally facilitated transactions into their conceptualization of e-Commerce. Participants representing utility companies were unsure whether electronically paid bills should be considered e-Commerce because customers are not placing an order for a good or service. Instead, in this industry, customers use the service and then pay after. Companies that earn revenue through leasing and non-retail companies that sell products or services by contracts (like the oil extraction company are also uncertain whether these transactions should be considered e-Commerce. The final contract and payment may be taken electronically, but negotiations may happen through other means and final payment may be made via check.

Two participants called for the need for industry-specific instructions to assist them when answering questions about e-Commerce. One of the utility company participants began to question whether they should be counting their operations as e-Commerce after further discussion. This participant wanted more industry-specific instructions to help them better understand whether to report their revenue as e-Commerce. One bank noted a similar need for industry-specific wording because the revenue-generating activities for their bank did not map well onto their concept of e-Commerce.

“Maybe that is where more clarification about our specific industry would be helpful. We have residential revenue amount, but is all of it done via e-Commerce? I don’t know. We would have to dig in and look into that for the future. Now that we have discussed that, maybe this does count, but it isn’t in the traditional sense.”

“Well, when you say “activities” what do you mean? Any account that is opened, the origination point is tracked so we know where it originates [online, by phone, physical location]. There are service records as well. We know how many online interactions, how many phone interactions…– when you say “transactions”, it means a lot of things in the banking world. If an account was opened or originates with our client care center, which is our telephone banking area, we can see that. If it originated online, with assistance of our client care center, we could see that. We would identify revenue for those accounts as belonging to electronic.”

Applicability for businesses that operate with salespeople

Another important consideration for defining e-Commerce is commission-based sales or whether salespeople are involved in the transactions. When salespeople are involved, some businesses do not classify those transactions as e-Commerce, even if the sale originated online. Others questioned whether transactions that a salesperson entered electronically into an application or online system should count as e-Commerce or not. The archetype of stores such as Amazon as e-Commerce can prevent participants from clearly mapping their business processes to their understanding of e-Commerce. These participants consider e-Commerce transactions as only those where the customer is interfacing directly with the electronic ordering mechanism, typically a website.

“We will review the orders from our website and coordinate the shipment with our local stores that have the inventory of the item as we received the payment from our website. If the customer orders an item online but then decides to visit our local stores to look at the item and complete the payment, then we will NOT count this transaction as e-Commerce and will select the sales-rep of the store who completed the transaction for the sale.”

“We do not offer a website to our customers to buy our products. Our marketing department has been handling the business sales of our products. They work out sales contracts with our customers.”

“We do have a very small [sales amount] through Shopify [which I consider e-Commerce]. But there is a lot of electronic sales [from the sales reps]. Sales are made via email and then logged electronically. But also [sales rep can] go in person and can make the sales in person and log it in electronically. They will go in the system and put it into the plant to fill the order.”

“No [I don’t consider our wholesale customers e-Commerce]. We have a customer service team that calls the customers and enters it directly into SalesPad.”

Sales mediated through salespeople are not currently explicitly included in Fayyaz’s (2018) definition, and most businesses using salespeople would not include these transactions as e-Commerce. However, some question whether they should be included when the salespeople use digital means for placing the orders.

4. Scope impact on e-Commerce definition

Another interesting aspect that arose from conversations about defining e-Commerce is that the amount of e-Commerce business activities a business engages in directly impacts how participants define e-Commerce. The impact of scope on the definition of e-Commerce occurred at both ends of the spectrum—both those with very little e-Commerce activities and those who could consider all revenue as e-Commerce. See Exhibit 3-1 for the amount of e-Commerce activities reported by participants.⁵ Those with very little e-Commerce activities could be confused about whether their business engages in e-Commerce and may omit activities that they do consider e-Commerce simply because they make up such a small amount of total revenue. Conversely, other businesses consider all of their revenue e-Commerce and can struggle to define it because there is no business need to understand what is and is not considered e-Commerce. Instead, for the AIES e-Commerce question, they would report all of their revenue.

“[I forgot about curbside pickup] because it is so small. It was a bigger thing during COVID, and while it is still offered today, it is used minimally.”

Exhibit 3-1. Amount of e-Commerce Activities Reported by Participants

Scope of e-Commerce	Number of Participants
All	5
Large	8
Medium	6
Small	5
None	7

One participant believed during most of the interview that her business did not engage in e-Commerce because it made up such a minute part of their total sales. However, when she saw the current AIES question, she did remember reporting the business’s eBay sales as e-Commerce. When thinking about e-Commerce more broadly, she did not initially include these sales because the majority of their business happens as in-person transactions. The impact of scope on a business’ conceptualization of e-Commerce may impact measurement error when small parts of the total revenue are omitted because of limited salience for respondents. Section 3.3.3 will include further discussion of businesses that consider all of their revenue e-Commerce.

5. Does the business conduct e-Commerce activities?

As a part of discussing each participants’ definition of e-Commerce, they were also asked whether the business conducts any e-Commerce activities. As detailed in Section 2, all businesses sampled for the e-Commerce debriefing interviews indicated some amount of e-Commerce revenue in the most recent AIES. However, seven businesses stated during Module 2 that they do not have any e-Commerce activities (see Exhibit 3-1). Two participants changed their mind when later viewing the current AIES e-commerce question. Another participant was hesitant about whether they would count their business operations as e-Commerce and had not made a final decision for future reporting by the conclusion of the interview.

When preparing for these interviews, we anticipated that some businesses may reject the idea that they are engaging in e-Commerce; however, we were surprised to find that this was the case for nearly a quarter of the completed interviews. Some of the confusion from these participants comes from the industry-specific concerns outlined previously. Notably, all participants who participated in this study indicated they were the ones who responded to the AIES. It remains uncertain whether their inability to recall their affirmative response about the e-Commerce activities in their AIES reports is due to delegating the question to another colleague within the company or whether there are other underlying factors at play (such as data entry errors). Later sections will continue to detail how these participants navigated their understanding of e-Commerce and whether their business engages in it.

3. Review of the Current AIES e-Commerce Question

Review of the current AIES e-Commerce question was included as the last module in the interview protocol; however, feedback about the question and its instructions corresponds closely with how participants’ defined e-Commerce themselves. Participants were first shown only the question text and asked about comprehension, then were shown the instructions (see Appendix B). This section will outline reactions to the question text alone as well as to the question text with additional instructions. The later sections will outline how comprehension of the current AIES question impacts ability to report e-Commerce.

1. Reactions to question text without instruction

When shown the question text alone (see Exhibit 3-2), participants most commonly described the question as asking about electronic sales, including using phrases such as “online,” “electronic channels,” “internet,” “website,” “ordering on an app,” and “virtual.” Most participants focused on the point of sales or the way a customer placed an order, similar to their own spontaneous definitions of e-Commerce. Some participants continued to define e-Commerce by what it is not (i.e., not in-store or brick-and-mortar purchases). A key phrase for participants was “that were ordered,” which communicated the idea of customer orders through electronic means. Some participants also highlighted that the question asks for a subset of total revenue, also described as “income.”

“Of the entire amount of revenue that our company made in 2023, how much was ordered through the internet and was not purchased inside a brick-and-mortar store.”

“The question asked about the total revenue from the product sales from a website.”

“Asking for the dollar value proportionally of the total amount of sales that were generated from electronic sales.”

Exhibit 3-2. Current AIES e-Commerce Question Text

The wording “ordered” played a big role for one participant (see Section 3.2.2). This participant originally focused on the delivery method of goods as a key feature of e-Commerce. After reading the question text and seeing the wording “ordered,” she was open to including any transactions where the product was ordered online, regardless of delivery method.

“To me, I would probably have answered with the items from the mobile pay. Because it was ordered online or through DoorDash, which is online, it’s not in person. To order… it doesn’t say delivered, it does say order.”

From the phrase “sales, shipments, receipts, or revenue,” participants focused on either sales or revenue. No participants mentioned shipments or receipts when describing their understanding of the question. In fact, “shipments” may have contributed to some misunderstanding from a participant who included remote delivery as key to her understanding of e-Commerce. This participant specified that she thought the question was asking about “how much was done through online orders, how much was paid through electronically, how much was shipped to someone where you may not know where they are located.” The phrase “shipments” may have reinforced an understanding of e-Commerce as including only transactions where goods are delivered remotely. Another participant also verbalized a dislike of the word “receipts,” sharing, “Receipts is usually bringing in stuff and that wasn’t included at all.”

“This seems like it is asking what is the total sales for online goods.”

“Asking me to separate the sales that are e-Commerce-related from the sales that are the total revenue.”

“I focus on the first part of it with the sales, shipments received and revenue reported. That ties in with your goods, services…. I focus more on that in terms of just meaning overall sales.”

“I would say - anything bought online or bought online and picked up in store. I would think to include that revenue.”

“I interpret this [question] as revenue. I was looking at the revenue that came from [online retailer].”

Another key phrase for participants was “controlled or coordinated electronically.” This phrase was key for one non-profit to understand that the grants they received should be counted as e-Commerce. Although initially this participant believed the non-profit business did not conduct any e-Commerce, she changed her mind after reading the question text and remembered that they did report their grant funding as e-Commerce. Another participant summarized the question as asking about “electronically controlled revenue,” highlighting this as a key phrase to her understanding. One participant liked the specificity of this phrase and said it improved clarity over asking about e-Commerce in general.

“Some of the [grant sources], they don’t mail us a check, you have to go online to a portal and connect that to an account so when they issue a grant, they sent it ACH into your account. For that reason, we did put a number for the answer to that question. It’s not because we provide something online it is because how we receive it…I believe this is the question I was just telling you about, we did put a number in there because it says at the end coordinated electronically, and for some of these grants that is how we receive it.”

However, the phrase “was controlled or coordinated electronically” also contributed to some confusion and misunderstandings from participants. One participant stated throughout the interview that her business did not have any e-Commerce activities.

“None of our goods, services, or products would be controlled or coordinated electronically, even the hotel reservations which would be online. It would still need to be coordinated and controlled in person when someone checks into the hotel.”

This participant did not understand that reservations made online hotel should be considered e-Commerce. Instead, she interpreted the in-person transaction as violating the definition provided in the question, because the payment was not controlled and coordinated entirely electronically. Another participant also noted that the phrase was confusing.

After reviewing only the question text, six participants thought the question was too broad and confusing. One participant felt confused when seeing a reference to “manufactured products” when they do not manufacture anything. Another participant recognized the question needed to be broad based on the wide variety of businesses that would respond to the survey, sharing “I know it is a very broad statement because not everyone does the same thing or collects funds for the same thing.” However, others felt it was so broad that anything could be included as e-Commerce, which could potentially lead to measurement error. This reinforces the idea of the “moving line” of e-Commerce; when participants started to think more about what could be considered e-Commerce, some also started to question whether all their revenue should be considered as e-Commerce.

“The question is so broad that it is going to be very difficult for anybody to say something isn’t eCommerce if you’re thoughtful, as I’m trying to be here.”

“So, to me, because of how it reads, this could be considered ANYTHING that was an order processed electronically, whether it was in our systems, or if they called us, or if it was online. Which is unfortunate, for e-Commerce, it seems like it’s something that’s solely online sales.”

Finally, as a positive, two participants expressed that they liked that the question only asked for industry totals rather than by location. They both stated that it would be extremely difficult and time-consuming to respond to the e-Commerce question by location. As will be discussed in later sections, some of the difficulty in reporting by location stems from lack of records because many businesses do not tie their e-Commerce activities to a specific location.

Industry-specific concerns

Participants reporting for businesses in industries that do not fit neatly into their conceptualization of e-Commerce want industry-specific instructions and wording to help them understand the question better. One participant, whose business offers services that are reserved online but delivered in person, struggled to understand that this source of revenue should be counted as e-Commerce based on the question text alone.

“I am not sure if that [online bookings] would qualify. I don’t know as it is confusing to me. None of our goods, services, or products would be controlled or coordinated electronically, even the hotel reservations which would be online. It would still need to be coordinated and controlled in person when someone checks into the hotel.”

With “services” buried in the middle of the question around a lot of other text, one participant missed it entirely, sharing “I think [the question] is [about] more manufacturing things, selling of things, not providing a telehealth service. I just don’t think it pertains to us.”

Another participant, whose business provides digital software, questioned whether their platform would be considered goods or services or manufactured products. He was unsure whether his business fit into the definition of e-Commerce provided in the question text, even though earlier he answered affirmatively that his business engages in e-Commerce. This issue is compounded by the fact that all of the business’s revenue could be considered e-Commerce, which makes the question more challenging to understand. In the end, the participant was convinced by the second half of the question text, including the phrase “movement was controlled or coordinated electronically” and decided he would report all revenue as e-Commerce.

“I think it needs another [statement] that would be “electronically provided services or physically delivered services” if that is the intention. That’s the whole thing—I don’t know the intention of the writer, but I know that I’m left out. So, I’m guessing its whether you’re interested in my revenue or not for this question…”

As outlined earlier, these participants struggled with comprehension of the second two prongs of Fayyaz’s (2018) definition of e-Commerce which includes digitally facilitated transactions and digitally delivered transactions. Some of these comprehension problems were improved when participants viewed the additional question instructions; however, other issues also emerged.

2. Reactions to question text with instructions

After responding with their understanding and comprehension of the question text alone, participants were then shown both the question text and instructions (see Exhibit 3-3). After reading the instructions, one-third of participants thought they changed the meaning of the question. In other words, when viewing the question text alone, participants were thinking about a different definition of e-Commerce than what was specified in the instructions. Most who said the instructions did not match their understanding of the question missed specific transactions that should be included as e-Commerce, such as BOPIS or monetary donations. One participant drastically changed her understanding of e-Commerce after reading the question text and instructions. As detailed previously, this participant initially believed that only mailed online orders would be considered e-Commerce. After reading the question text, she believed that all “online orders” should be included, but this was further confirmed when she read the detailed instructions. She incorporated this new information into her definition of e-Commerce and would report about all online orders, rather than only those that were mailed.

Exhibit 3-3. Current AIES e-Commerce Question Text Instructions

Additionally, after seeing the instructions, eight participants commented they were helpful because the instructions provided further clarification and confirmation about what the question text alone asks for. Two participants appreciated the specifications of what to include but thought the question could also benefit from listing out what to exclude. One participant whose non-profit organization provides services found the BOPIS instruction confusing. She thought it would be more helpful to have what to exclude specified rather than what to include.

“The detail at the bottom gives me more details about e-Commerce or e-shipment. They make more sense now, and they matched my understanding.”

“Matches. I think it drills in…the main emphasis is online, online, online, online, so to me, [the instructions] just drive it home.”

“It matches what I was thinking about, but it provides a lot of clarity on it. It is more descriptive of buying online, what buying online means, shipments…it gives it more clarity on everything you are manufacturing or delivering.”

“I think it is the top bullet point there that is really applicable for us: “revenues from customers entering orders directly on their own or a third party website.” That really helped confirm the information we then reported.”

“When it says pick up in store, so again, does it really include if membership is done online but they come into the office to pay? I’m not sure if that counts. If they give examples, like if they paid with check or cash that should not be included. It would be great to have what should be excluded. It has here what should be included but not what should be excluded. That would make it a little clearer.”

“I think that your includes as part of the instructions now could also have an excludes. I would have written it as “Examples of what to include” or maybe not, but certainly examples of what may be excluded.”

While the instructions were helpful for most participants, five participants found them confusing and wordy.

“This is not helpful… The first paragraph is way too complicated. Again, the survey should be easy for people to understand. This gets into the highly technical information that you want.... Then there is a lot of things here to include.”

“Some of [the instructions] clarified it a little bit more, and others I was like that still doesn’t make sense. So, I’d have to call [Census]. I would just call to make sure that we are understanding it correctly so we didn’t report the wrong figures.”

One participant specifically experienced some confusion about the phrase “was controlled or coordinated electronically” in the question text when read in combination with the instructions. This participant started to question where the “line” was for e-Commerce activities and whether credit card payments should count as e-Commerce.

“The last portion ‘that were ordered or whose movement was controlled or coordinated electronically.’ To me that is confusing because when I think electronically, my brain goes to the swipe for the credit card, so does that include that or no? If I focus more on the online, online, online or the mobile portion of it that helps me with clarity of it versus electronic. There are a lot of electronic functionalities in what we do here that would not count as e-Commerce. To me, I would focus more on online vs in house—that helps me segregate those out.”

Overall, feedback about the question instructions is mixed, with some positive and negative reactions. Next, we will explore specific feedback about the wording and phrases used in the instructions. The rest of the section will outline participant reactions to specific phrases used in the instructions. This feedback may help to better understand how participants’ conceptualization of e-Commerce does and does not match the current AIES question wording.

Payment may or may not be made online

Nine participants commented on the instruction “Payment may or may not be made online (e.g., order online but pay at a location in person).” This sentence stands out to participants because it contradicts the dominant understanding of e-Commerce. As discussed in Section 3.2.1 almost all participants considered e-Commerce transactions to include those that are mediated through an online website where payment is conducted remotely and independently of any physical business locations. The instruction states that payment may be made in person after an order is placed online, which violates the common perception of e-Commerce as remote electronic sales. Five participants found the instruction confusing, with several indicating their surprise that payment in person could be considered e-Commerce. One participant discussed an example of making a reservation online for a restaurant, but then eating in person at the restaurant and paying.

“I think if you order food and you sit down and eat it and you don’t pay for it [online] then that’s not e-Commerce. To me e-Commerce is more you’re ordering, you’re paying, most instances you’re receiving the product online. The one caveat would be curbside or in-store pick up, the convenience is you’re doing everything electronically.”

The idea of paying in-person for a service provided in person violates common understanding of e-Commerce. Many participants struggled to incorporate this instruction into their conceptualization of e-Commerce. However, four participants acknowledged that they could consider these types of transactions e-Commerce but initially would not have thought of this as e-Commerce when thinking about e-Commerce generally nor when reading the question text alone.

“I think it is a little bit more clear in that, for instance, the line that says “payment may or may not be made online” that is how our [business services] are handled. You make a reservation, but I think you end up paying at the location in the retail store. So that specifically under this guidance would be included as e-Commerce. The order was secured via electronic means but you’re just remitting payment in person. [It’s helpful.] It’s specific to our business, it’s something that we utilize…[But] I think when I read and was looking to complete the survey, my mind immediately went to the platforms I know we have, and not really diving into the nuance of, from the retail space this specific amount relates to [business services] that would have been reserved on this platform…”

“Based off your definition here, it counts. But initially I would not have thought of [making a sale online and paying in-store] as e-Commerce, without reading the definition. I would have thought no.”

Three participants indicated that their business does not have the ability for customers to pay in-store after placing an order online. Online orders require payment at the point in time the order is placed. The instruction did not apply and left them confused about when it could be applicable, especially because the instruction did not align with their conceptions of e-Commerce.

“We don’t have that option. A customer can’t make tender in a store but it ships online. Everything would be done online.”

“Payment may or may not be online. To us payment has to be online, otherwise you are going into a store.”

This instruction about payment being made online or in-person violates how participants’ conceptualize e-Commerce. Although the first prong of Fayyaz’s (2018) definition states “digitally ordered transactions (i.e., sales conducted over computer networks),” participants generally only consider e-Commerce transactions when the point of sale occurs electronically. That is, they consider online payments e-Commerce more often than in-person payments for online orders. The current instruction does not align with participants’ understanding of e-Commerce nor with the question text, which may lead to measurement error.

BOPIS

Related to the instruction about payment that may or may not be made online, participants also reacted to the instruction “Buy or reserve online, pick up in store (BOPIS).” Participants reporting for businesses that provide services highlighted the wording “reserve online” when thinking about how the instruction applies to their business. Like the findings about the phrase “Payment may or may not be made online,” the idea of reserving the service online but paying in person violates many participants’ construction of e-Commerce, which may lead to measurement error.

For example, one participant believed that providing services in person did not count as e-Commerce, even when payment is provided online in advance. She understood curbside pickup as e-Commerce and if her business had any curbside pickup transactions, she would include those in her reporting. However, the defining distinction to her was the difference between goods and services. She believed that receiving goods in person could be considered e-Commerce but providing services in person could not be e-Commerce.

“To me [that instruction is] I had to order online and I had to pay online, that’s what this is saying. If you’re conducting business via an online mechanism, portal, whatever it might be. So, the person who walks into our office and and gives us their credit card for fifty dollars, to me, that would not be e-Commerce… We do [take payment through website] but not for all of our [business services]. The only example of services we provide that we take payment online and deliver the services online is if we did a webinar that they could pay online and services delivered online as well. But for the other example, you literally have to come into our office. All you would be doing is paying via credit card. We are delivering services via face-to-face delivery rather than a product that you could ship to your house or pick up online or at the curb.”

Relatedly, another participant whose business provides services debated whether reservations in advance should be counted as e-Commerce because customers have not committed to any payment or service until they show up in person and are charged. He stated that even with the instructions, he would respond to the question with $0 revenue since he did not believe that these types of transactions constituted e-Commerce.

“It’s a grey area, if you say that if someone orders, by that I mean they go online and get a reservation and know it’ll be $22 but they haven’t committed to anything until they show up. We aren’t going to charge them anything because there has been no service provided. If this was asked today I would say zero.”

As mentioned in Section 3.2.2, this participant made a distinction for e-Commerce between payment in-person and electronically stored credit cards. When their business is able to store credit cards on file and charge the cards automatically for services provided, they would consider that e-Commerce. Currently, the business takes reservations online and completes the payment in person when the service is delivered. This participant struggled with the instructions about reserving online to pick up in store because it violated their understanding of e-Commerce. Another participant stated candidly that if the customers are paying in store, they would not consider it an e-Commerce sale.

These three participants highlight a concern for measurement error when participants read the instructions, but they so thoroughly violate the participants’ understanding of e-Commerce that they may choose to misreport. The way businesses organize their records also impacts reactions to the instructions and whether participants can incorporate the specific instructions into their understanding of e-Commerce. For those who do not track in-person payment for online orders, it is more difficult for them to understand the instruction and, furthermore, almost impossible for them to report on. Further detail about the lack of records reflecting in-person payment for online orders will be discussed in the next section (Section 3.4).

However, one participant found clarity in the instructions and adjusted his understanding of e-Commerce to prioritize how the order was placed and deprioritize how the payment was made. He found the BOPIS instruction helpful to construct a new, broader understanding of e-Commerce.

“[For instances where the [business service] is reserved online and the person comes in and pays in cash,] you are securing the product or service via an electronic means, you’re just remitting payment physically. I would still probably consider that e-Commerce.”

When he focused on in-person payments as just how the payment was “remitted,” he was able to expand his definition of e-Commerce to include both in-person and online payments. This participant also explained that he was able to report these transactions separately because of the way the records are organized around separate POS software systems. It was easier for him to incorporate this idea of in-person payments because the records the company keeps matches this conceptualization.

Electronic Data Interchange

Another phrase that was confusing for three participants was “Electronic Data Interchange (EDI).” For one participant, it was unclear whether an EDI network was the same as the online portals they access to move money. Another participant wondered if the business had an EDI or not, seemingly unfamiliar with the term. One participant was also confused about what an EDI was and thought there should be additional details to explain the term. This term may be dated and not relevant to current business operations.

“In the paragraph right below where you would put a number, ‘an Electronic Data Interchange (EDI) network or a private network’ we do have that in a way. We aren’t buying or selling products to people online, but we do have online portals where money is moved around, but I don’t know if that was the intent.”

“I am more confused after reading this than before… an Electronic Data Exchange Network? Do I have one of those?”

Shipments to other plants

Two participants also identified the statement “Shipments to other domestic plants of your own company for further manufacture, assembly, or fabrication” as confusing because these transactions are not typically considered revenue-generating and thus are not tracked.

“I wouldn’t think of the shipments to other domestic plants as being e-Commerce. I would say the second bullet point doesn’t really make sense to me… it wouldn’t be considered revenue, so it is not tracked. It wouldn’t be something I have access to; the location level would have to provide that if there was anything.”

Additional consideration and research should be given to this instruction to ensure that it is working as intended. The question text asks for a subset of total revenue, but if shipments to other plants are not considered revenue, then this instruction should not be included in the AIES question.

Kiosks

One participant was very confused about how to consider kiosk transactions in the context of e-Commerce. Her business offers fast casual food and includes kiosks in the store; however, she was not confident that this should be considered e-Commerce. Furthermore, there would be no feasible way for her to report kiosk transactions because these are considered in-store sales. More details will be reported in Section 3.4 when discussing record-keeping practices.

“Actually… you know, we have kiosks. So, if they go to the front of the store and purchase through the kiosk, I’m not sure that’s considered e-Commerce… it says buy online with kiosk… if there’s a kiosk in the store and you can swipe your card… I wouldn’t know how to answer that. It says buy online, is it considered online if you are 3 feet from the front register? That one would require more information. If you’re in store and doing it through the kiosk, is that online? Or online and away from the store?”

Differential inclusion of kiosks in reports of e-Commerce revenue may lead to measurement error, but this error may be unavoidable if business records do not allow for easy reporting.

m-commerce

One participant thought the phrase “mobile devices (m-commerce)” was confusing, stating, “What is ‘mobile devices (m-commerce)’ – is that like cell phone usage or iPad? I don’t know what you mean by mobile devices here.” Another participant felt the inclusion of m-commerce needed further clarification on whether orders placed by phone should be included as e-Commerce.

“The only other thing I would say here, because of the inclusion of mobile devices, maybe you need something about clarifying a “call-in” order as opposed to “ordering on your phone” or is that something that would be included here? It is just a point of clarification I would need.”

m-Commerce may have been a more prominent term in the past, but today m-commerce can cause more confusion than clarity because mobile phones are used seamlessly for online transactions.

Monetary donations

Four non-profit organizations shared their reactions to the instruction “Monetary donations received online as revenue from electronic sources (for tax-exempt firms).” Three participants did not initially include monetary donations in their conceptualizations of e-Commerce. One participant received most of their revenue via grants, which they had not initially considered e-Commerce. The instructions helped them understand that grants paid through electronic funds transfer (EFT) should be included as e-Commerce. The other participant who did not initially consider monetary donations as e-Commerce shared the following:

“I wasn’t thinking of monetary donations. That makes it clearer that we need to include that. I think we only picked up the credit card transactions, but maybe it included the [donations].”

However, this participant also had a question about the monetary donations, wondering, “we get donations via stocks… I’m thinking does that apply here? … if we can get more break down for the non-profit, that would be great.” This participant was confused whether other forms of donations should be included or only those made via direct donation.

Conversely, another participant thought that monetary donations were immediately obvious as e-Commerce in the non-profit space.

“For me personally, [monetary donations are] something I immediately associated with [e-Commerce]. We don’t have e-Commerce like a retailer that has an online store like Walmart, but I kind of immediately, when I was thinking about e-Commerce thought about ways people can make contributions online, and that would be e-Commerce for us. So, this [instruction] is really codifying that, but that’s an assumption I had from the beginning.”

However, this participant was previously employed in the for-profit sector and may have a higher level of comprehension than other reporters for non-profit organizations. The risk of measurement error for non-profit organizations is very high if they do not include electronic donations in their report of e-Commerce. The fact that the instruction is hidden may lead to a higher rate of issues since non-profit organizations may read the question text alone, respond $0, and move on to the next question without reading the instructions. The placement of the instructions will be discussed later in this section.

Industry-specific applicability considerations

When reviewing the instructions, several participants called out aspects of the instructions that were not applicable to their business, such as instructions about monetary donations, curbside pickup, manufactured products, and kiosks. Although some participants continued to acknowledge the need for diverse economy-wide instructions, others felt confused and that the question was not applicable to their business at all.

“A lot of those subcategories are not relevant to us. We deliver, customers come and pick up, but we don’t have a kiosk that I’m aware of… The survey is for a lot of businesses, I realize not everything is going to be for us. I do what I can do.”

“I think there are things that are just not relevant to us. Like we don’t have a kiosk/locker/tower pickup option, but that doesn’t mean it shouldn’t be there because it is probably relevant for someone else. We don’t do monetary donations from customers. I think it is fine and I would just skip the pieces not relevant to us.”

“Down there in the bottom where it says include these things, we don’t fit into those categories.”

“I think it is more manufacturing things, selling of things, not providing an online [business service]. I just don’t think it pertains to us.”

The instructions provided seem to over-represent retail-specific e-Commerce activities, which may come at the expense of other industries, such as utilities, and service industries, such as hotels and telehealth providers. After reviewing the instructions, four participants continued to state they did not engage in e-Commerce and felt the question was not applicable to their business at all.

However, reviewing the question text and instructions made two participants change their mind about whether their business engages in e-Commerce and one participant question whether their business engages in e-Commerce. Both of these non-profit participants originally did not think of donations and grants as e-Commerce when probed unconstrained about the definition. However, after reading the question text and instructions, both believed that grant money received electronically (e.g., through EFT, through Payment Management System, through ACH) should be considered e-Commerce. They also both recalled that they had reported these grants on the AIES e-Commerce question this year. When thinking about e-Commerce in general, these participants did not consider non-profit business activities such as receiving grant money as fitting into the concept of e-Commerce, which could lead to significant measurement error. Furthermore, one participant’s organization only generates revenue through grants while another’s organization generates 95% of revenue through grants. This highlights the potential need for industry-specific instructions that would appear outside of additional help text to assist businesses like these in correctly responding to the e-Commerce question.

The last participant who questioned whether their business does or does not engage in e-Commerce after reviewing the current AIES question was confused about how to apply the instructions to their specific industry (utilities). As detailed in Section 3.2.3, this participant was uncertain whether participants going online to pay bills should be considered e-Commerce when asked about e-Commerce broadly. After reviewing the question text and instructions, this participant continued to try to understand e-Commerce in the context of their industry.

“Now I am not quite sure if I should report e-commerce for our company…I still don’t consider our company engaging in e-commerce. But if we have to report [based on these instructions], it will be ‘all or nothing,’ either our total revenue is considered e-Commerce, or none is e-Commerce.”

This participant was unsure how to apply the archetype of e-Commerce (i.e., Amazon) to their specific business. They were unable to come to a definitive decision about whether they would report e-Commerce revenue and thought that it may be appropriate to report either all of the revenue as e-Commerce or none of it. They wanted further industry-specific instructions to help them understand, sharing, “None of the instructions really fit the description of our company’s business transactions. We would need some instructions that fits our industry for us to follow and report.”

The other participant reporting for a utility company had similar questions about the applicability of the question text and instructions to their business. They were unsure whether instances when customers reported meter readings online should be considered buying or reserving online. They were also unsure how to interpret “placing an order” in the context of their industry.

“I just read that second bullet, but I don’t think that one is really applicable. ‘buy or reserve online’—it’s interesting, right. The gas comes through the pipes and the end users like residential folks for instance can pay online, so I guess that is them buying online, but it is interesting. It depends on how you view it…. I guess I still don’t know if a customer enters their meter read online—does that count? It is what is going to generate the invoice… It seems like it would be based on ‘buy or reserve online’…and they pay online. So, in a way, yes. It is just a little tricky for the gas or utility industry. It’s just different… [the instruction] says ‘sales of goods or services’—I mean gas would be considered a good. It almost seems like it isn’t applicable to us. Is putting a meter amount of what they used placing an order? I don’t know. Is that what is being meant there?”

This participant went on to ask for further industry-specific clarification, sharing:

“I don’t know if the Census Bureau could clarify for our industry—I would think that across the industry things are fairly consistent in how transactions are recorded as far as revenue and customers.”

They also raised the potential for measurement error in their industry if different businesses interpreted transactions as either e-Commerce or not e-Commerce. Other participants also raised the need for industry-specific instructions, such as one participant, who shared, “I don’t know how you’re going to change all the instructions you’d have to get industry specific to make it fit for every business.”

Other industries that may benefit from industry-specific instructions include hotels and telehealth providers. This participant thought throughout the interview that their hotel business should not be considered e-Commerce, and when asked about how to make it clearer that the hotel reservations should be included, she stated, “I would include reservations or something like that. Online reservation or hotel booking so then we would include it.” Comparatively, another participant questioned whether telehealth services should be considered e-Commerce when discussing e-Commerce at large and changed her opinion after viewing the question text and instructions. She believed that “if [telehealth] is part of [e-Commerce], then it has to be in the instruction,” otherwise she would not include it in her reporting.

The current AIES e-Commerce question instructions seem overly focused on retail industries and may omit key instructions for other industries, as outlined in this section. Further consideration of industry-specific instructions may be important to minimize measurement error.

Applicability concerns for business activities operated under contracts or with salespeople

As noted in Section 3.2.3, businesses that use salespeople also struggle with conceptualizing e-Commerce in the context of their business. After viewing the question instructions, one participant described the process for salespeople.

“Our salespeople are out visiting a [client site] and they put the order in their app or iPad or device, they are traveling to the place where the customer is in person and get credit for the sale. Is that really e-Commerce? That person is making the sale in person, but it’s put into the app…To me e-Commerce is based on how the sale is made, how the revenue is created not how received.”

This perspective is similar to another participant, who after seeing the question instructions believed e-Commerce should be defined by POS rather than how the payment is remitted. For this participant, POS would be with their salespeople regardless of how the sale is recorded in their system. Because customers are not interfacing directly with an online or electric system, they would not consider these transactions e-Commerce.

The other participant whose business uses salespeople also questioned how to categorize these transactions. For their business, customers can make a reservation online, but they travel to the store in person to view the product and make the final purchase. When this happens, the salesperson interacts with the customer and may make a difference in the final products purchased. They would not consider these transactions e-Commerce.

Another business-specific aspect that participants brought up was contracts completed electronically and sales made over email. One participant described the electronic contracts they use with customers.

“I mean, everything happens in contract, and we have contract emails and people sign it electronically. I hope this is not part of the e-Commerce. Because if it does, it will be a different story. In this day and age, everything happens on the Internet. So, we do not consider contracts as e-Commerce activities.”

Another participant reacted to the phrase “places an order, or the price and terms of the sale are negotiated over the internet,” stating that their business does engage in negotiations via email, but they would not consider those as online. This participant shared, “Well, [the instruction] doesn’t exactly match our system… I would understand that, but email goes into our retail store, so we don’t consider those as being ‘online.’” This participant did not consider email orders as e-Commerce. Including this point in the block of text with many different conditions made it more confusing and difficult to understand.

These participants did not believe that transactions mediated through other electronic methods should be considered e-Commerce, including salespeople placing an electronic order, online contracts, and email negotiations or sales. These methods of sales did not match with their conceptualization of e-Commerce, and they would not report these as e-Commerce revenue. This was the case even with further instructions about negotiations over email because their records would not allow for easy reporting of this POS regardless.

3. Scope impact on e-Commerce definition

As detailed in Section 3.2.4, scope continued to have an impact on how participants understood the concept of e-Commerce and reacted to the question text and instructions. The participants who consider all of their business revenue e-Commerce felt confused about having to report all of their income twice, once for company-wide income and a second time to indicate all income is e-Commerce. The e-Commerce question specifies “of the total amount of Sales, Shipments, Receipts, or Revenue reported,” which communicates to respondents that they should report a subset of their total revenue. When their entire revenue is e-Commerce, participants struggle to harmonize this unstated parameter of the question. For example, one participant did not want to report the income twice, even though she considered it all e-Commerce. Instead, she would report $0 of e-Commerce revenue because “revenue is revenue.” Interestingly, she thought she would make a distinction depending on the placement of the questions.

“I would probably look to see where it falls in the questions. If the brick and mortar, is the first question and the eCommerce revenue is the second, then the brick and mortar would likely get the revenue. If it were reversed, then eCommerce would get the dollars.”

Another participant, whose e-Commerce activities comprise the majority of all revenue, felt confused seeing this question. As outlined earlier, this participant felt the instructions were too expansive and she started to wonder whether all of her business’ activities could be considered e-Commerce.

“[It’s confusing] that it talks about revenue from electronic sources, but then asks about everything! I just probably would put the same figure there [as all revenue]. That would be everything…this makes it sound like all sales. Again, the same way, it’s a different way of asking the same question you already asked, which is frustrating. Is that the same as all sales? Why don’t you say all sales again? Everyone uses an order system, everyone uses email, and email is electronic, so therefore it applies. Or if they take the order on their cell phone, that’s electronic.”

On the opposite side, businesses with a very small amount of e-Commerce operations also struggled with responding to the current AIES question. For example, one participant described earlier in Section 3.2.4, initially stated the business did not have any e-Commerce. After viewing the question, this participant realized that she did report an amount in response to the question on the AIES. However, the online sales this business made through eBay amounted to less than 0.2% of total revenue, which is why she did not consider it e-Commerce. For her, the word “e-Commerce” implies a large scope of business committed to electronic methods of sales. Her business includes such a small portion of e-Commerce that she struggles to map it to her conceptualization of what e-Commerce means.

As this section outlined, the current e-Commerce question can pose challenges for both those with very little e-Commerce revenue and those where all revenue could be considered e-Commerce.

4. Visibility of instructions

Although the instructions were useful for most participants and helped make the definition of e-Commerce clearer, some participants noted that they did not see these instructions when responding to the AIES. Twenty participants indicated that they did remember viewing the e-Commerce question when responding to the AIES. Of these 20 participants, only nine remembered viewing the additional instructions; some of these nine participants only assumed that they viewed the instructions, they were not positive.

“I did [look at the instructions]. Because “electronically” threw me off a little bit and I felt like I needed to drill in.”

“I think I have viewed the additional instructions to make sure that what I was thinking of as electronic was what Census Bureau was thinking it was.”

Four participants who did not remember viewing the instructions believed they were not available, with several actively looking for instructions for the e-Commerce question and unable to find them. This finding brings up the issue of visibility of the instructions when they are hidden behind an expandable question mark. To reduce measurement error and increase the consistency of comprehension across respondents, it may be important to consider presenting the instructions alongside the question text. Some participants who looked for the instructions could not find them, and others assumed they understood e-Commerce based on the question text alone. However, as demonstrated in this section, the instructions include many different aspects that respondents do not initially consider in their concept of e-Commerce from the question text alone.

“I don’t believe the other instructions were there.”

“I did not use additional instructions. I did not know we had them available.”

“I don’t think I even saw this. You said something about a question mark?”

“Oh, [the instructions are] very good. I don’t think they had that on this last survey, I don’t recall. I looked at every include/exclude. This would have helped. This is much better. It’s very specific. It pretty much defines it so you can get the right answer.”

“…the real question is, do you hide this behind the icon? And this is something we ask ourselves every day at my company, do we hide this and declutter the reading because not everybody needs it or do you show it all and it takes longer for them to think about their answer? I think it would be reasonable to not hide this because you could see that I was struggling to see if I was right or wrong. But that said if there was a little blue question mark thing, I would have clicked it on this one to get some confirmation that I was thinking the right thing.”

4. e-Commerce Record-Keeping Practices

As outlined in the previous sections, participants were first asked about their understanding of e-Commerce and subsequently probed about how they keep records of these e-Commerce activities. Because participant definitions were used to guide these conversations, there was a limited amount of data collected from participants who believed their business does not engage in e-Commerce. Therefore, the previous sections clearly outline participants’ definitions of e-Commerce, including their reactions to the current AIES e-Commerce question, to help contextualize this discussion about record-keeping. This section will include a report of record-keeping activities from participants’ own definitions of e-Commerce and the additional aspects of e-Commerce they considered while reviewing the AIES question.

1. Levels of e-Commerce records and tracking

Five distinct groups of businesses emerged from the data with different levels of e-Commerce activity tracking (see Exhibit 3-4). The first major distinction is businesses that do and do not believe they are engaging in e-Commerce activities. Those that believe they are not engaging in e-Commerce do not track any e-Commerce activities. Among those businesses that engage in e-Commerce, there are two important groups: (1) whether the point of contact is knowledgeable about e-Commerce and (2) whether there is a business need to track e-Commerce. For businesses where e-Commerce is a distinct revenue source (e.g., retailer with online and physical stores), tracking e-Commerce is crucial to monitoring business operations and making decisions. When e-Commerce is not key to the business (e.g., only a very small part of business is e-Commerce) businesses do not invest the resources to track it. Businesses in the last category consider all revenue e-Commerce and are de facto tracking their e-Commerce activities because they track their total revenue.

Exhibit 3-4. Types of Businesses Tracking e-Commerce Activities

Some interesting trends also emerge when considering these five groups by the size of each business. In this sample, large businesses were more likely to report e-Commerce activities and understand e-Commerce in line with the Census Bureau’s definition. Only businesses with less than 35 locations indicated that they do not engage in e-Commerce. Additionally, there are businesses of all different sizes that include a point of contact for the AIES who is not knowledgeable about e-Commerce and does not have access to the records for the business. Businesses that engage in e-Commerce but do not track operate at both ends of the spectrum, both small and large businesses. Finally, businesses that indicate all their revenue is from e-Commerce are more likely to have fewer locations.

These differences by business size correspond to some of the earlier findings about how the scope of e-Commerce activities influences the conceptualization of e-Commerce and comprehension of the current AIES question. It is important to note that this sample is not representative of all AIES participants; however, some of these trends may hold with other businesses, especially the idea that smaller businesses may be more likely to believe they do not engage in e-Commerce. Smaller businesses generally have fewer resources for record-keeping and tracking data and may also engage in fewer activities that match the archetype of e-Commerce (i.e., Amazon).

In addition to differences by size of business, there were also some differences between for-profit and non-profit businesses. Out of the eight non-profits interviewed, they fell into three of the types of businesses (as outlined in Exhibit 3-4), believing either they did not engage in e-Commerce, that all revenue was e-Commerce, or they engage in e-Commerce and tracking these activities (see Exhibit 3-5). There were no non-profit organizations that indicated they engage in e-Commerce but do not have access to the reports or that they engage in e-Commerce but do not track. Non-profit organizations typically operate on a smaller scale and with fewer employees, so it follows that there would be fewer AIES contacts that do not have access to the reports compared with for-profit businesses. Non-profit organizations must be efficient and assign those who work directly with the business’ data to report for the AIES.

Exhibit 3-5. Tyes of Businesses Tracking e-Commerce Activities by Non-profit versus For-profit Status

	Non-profit		For-profit		Total
Does not engage in e-Commerce	3	42.9%	4	57.1%	7
Engages in e-Commerce but doesn't have access to those reports	0	0.0%	7	100.0%	7
Engages in e-Comm but doesn't track	0	0.0%	2	100.0%	2
All revenue is e-Commerce	2	40.0%	3	60.0%	5
Engages in e-Commerce and tracks	3	30.0%	7	70.0%	10
Total	8	25.8%	23	74.2%	31

The rest of this section will focus on exploring the differences identified between these five types of businesses and share additional details about the ways e-Commerce records are organized for those businesses who do track these activities.

Contact not knowledgeable about e-Commerce (n=7)

One key finding from the e-Commerce interviews is the number of businesses where the point of contact for the AIES acts a liaison for their business without intimate knowledge of or access to the business’ records. Seven participants (23%) indicated they had limited knowledge about how the records are organized because they typically collaborate with others to pull the data. These participants who act as point people for their organization do not have visibility into the records their business keeps. Instead, they must be knowledgeable about who to ask for the data to report on the AIES. In these cases, question comprehension becomes key for this type of AIES respondent, especially for the topic of e-Commerce.

As illustrated in the previous sections, participants have a wide-ranging understanding of what constitutes e-Commerce. In cases where the point of contact for the AIES must act as a liaison, if they do not correctly understand what activities are included in e-Commerce, they may never even attempt to reach out to the correct colleague for e-Commerce data. These participants are the gate keepers for e-Commerce data, and although many do consult with others on the questions they review, others act alone to gather and submit the information. If they do not understand what e-Commerce is, there could be serious implications for measurement error, especially for responses that could be missing entirely.

Several participants who were not knowledgeable about the e-Commerce operations at their companies were able to make guesses about the way their records are organized but could not say with certainty. Four participants indicated they believed there was a business need to track e-Commerce activities, but all seven participants were uncertain how the e-Commerce records would be flagged in the data. All seven were also uncertain about the level of granularity available, but one participant guessed that the records would be organized by location. These participants were unable to provide many details about the e-Commerce activities at their businesses, but all of them did believe their businesses engage in e-Commerce. They recalled reporting e-Commerce data on the AIES, supported by their colleagues.

Businesses that do not engage in e-Commerce (n=7)

Seven businesses indicated that they did not engage in e-Commerce, as detailed in the previous sections. Two of these businesses realized they do track e-Commerce after being shown the current AIES e-Commerce question. These two non-profits originally omitted electronically transferred grant money and donations that should be considered e-Commerce. After adjusting their understanding of e-Commerce, both organizations indicated that they had records at the company level about total grant money, which they had in fact reported on this year’s AIES. A third business was uncertain whether they did or did not engage in e-Commerce after viewing the current AIES question. As indicated in the previous section (Section 3.3.2), this participant would choose to report all revenue as e-Commerce if they did decide their business engages in e-Commerce. The other four businesses that believed they did not engage in e-commerce did not have any records to report because they do not track any e-Commerce activities.

The businesses that believe they do not engage in e-Commerce typically have no records of e-Commerce activities unless all of their revenue could be considered e-Commerce. The lack of records reinforces for these participants the idea that they do not engage in e-Commerce, along with their conceptualization of e-Commerce. These businesses included a digital marketing company, a management company that included many different businesses, a tribal government that included a hotel, and a utility company. These businesses all operate in ways that do not fit the archetype of e-Commerce (i.e., Amazon), and they struggle to harmonize the provided AIES definition of e-Commerce with their conceptualization. These businesses also typically fit the second and third prongs of Fayyaz’s (2018) definition of digitally facilitated or digitally delivered transactions, along with some having online reservations for in-person services. As outlined previously, businesses with these characteristics struggle to understand that their activities could be considered e-Commerce.

No business need to track e-Commerce (n=2)

As outlined at the beginning of this section, the next major distinction for businesses is whether the business has an identified need to track e-Commerce activities. Two businesses indicated they do engage in e-Commerce but were not currently tracking e-Commerce activities because of a lack of business need to do so. One participant, representing a utility company, has been discussed extensively in the previous sections because of their confusion over whether their business activities should be considered e-Commerce. In the end, this participant did consider some of the activities e-Commerce, such as payments for bills online. However, the business currently makes no effort to track these activities as a distinct aspect of the total revenue because there is no business need to track.

“It doesn’t seem like [e-Commerce] is a key metric that anyone is interested in for our business specifically. We have a customer billing department. Maybe for them that is something they are focused on but we aren’t aware of it. For what we have historically reported on, being a public company, we aren’t specifically concerned with tracking [e-Commerce], and maybe if we were it is called something else. Or maybe it is just viewed as website payments internally.”

After reviewing the current AIES question, one participant was asked whether they could report e-Commerce records as the question defined them. The participant was unsure what records existed and how time consuming it would be to retreive them.

“I don’t really know how easy or difficult to get a report that says this is the amount that was paid online this month or this day. Maybe that exists. I’m not sure… it could be as simple as reaching out to our customer billing group and saying we are looking for the amount of revenue paid online and meter reads done online. But I don’t know how much time would be involved for us or the other department needing to provide it.”

Another participant indicated no business need to track e-Commerce reports for a management company that holds several different entities, including a golf course, movie theater, and manufacturers. This participant indicated that currently they do not identify e-Commerce activities at any of the companies, though some would have e-Commerce transactions. He explained that they would have to flag the data, either manually or automatically, sharing “We would have to go back to the initiation of that sale, and identify it that way through a flag in the data or a new account.” He indicated that “On a scale of 1–10 [with 10 as most difficult], it would be a 7 [level of difficulty] for all companies.” The company would have to fundamentally adjust the way they are tagging and managing their sales records to be capable of reporting about e-Commerce activities.

This participant was also asked how he could report e-Commerce activities for the current AIES question and explained that he would only be able to make a best guess and that it would be time consuming.

“I have to rethink my whole recording process on my transactions. I wouldn’t be able to give you accurate data. It would be a best guess at best…I could try to go to cash sales vs non-cash sales and go to [collect on delivery or] COD deliveries and how they make them. There would be ways to rule things out, but it would take quite a bit of time for each of the companies to come up with that. I would estimate 5 hours per company.”

Businesses without a current business need to track e-Commerce but that do engage in e-Commerce have minimal records at best about these activities and no way to disentangle e-Commerce records from total revenue. These participants would need to spend a significant amount of time working with colleages or manipulating the records to be able to respond to questions about e-Commerce.

Business need to track e-Commerce (n=10)

The largest group of participants were those whose business does have a need to track e-Commerce (as the business defines it) and thus have an easy way to parse out e-Commerce records from total revenue. The data included and excluded from their e-Commerce records map directly to their conceptualizations of e-Commerce (see Section 3.2). All of these businesses that track e-Commerce currently have one place for their records where e-Commerce transactions are clearly identified in the data, with flags such as “online,” “web,” or by the point of sale software system. Additionally, these records are all available at the transaction level and typically updated on a real-time or daily basis, though reports are typically reviewed on a monthly basis. All 10 of these businesses have both physical locations and an online presence, which maps well to their conceptions of e-Commerce. Eight of these 10 businesses have an invested business need to track e-Commerce as one of their multiple revenue streams. Most of these participants indicated that tracking was important to grow their e-Commerce revenue; however, two participants indicated that regulatory reasons are the primary motivation for tracking e-Commerce.

“[Tracking e-Commerce is] very important because we are trying to grow and we have grown significantly over the past few years our presence online for purchases. We look at online just as closely as our store sales.”

“[Tracking e-Commerce is] very important not only for the sake of revenue recognition but also looking at patterns and the type of people who are utilizing our online services.”

Another two participants indicated that there is a business need to track and identify e-Commerce records, but it is only secondary to tracking overall income and revenue.

“I would think that there is a [business need to track e-Commerce] in the extent of looking at the amount of, say memberships, revenue that we generate from people enrolling in online memberships, from a fundraising standpoint. We had this amount in 2022, this amount in 2023, where is this difference arising?...there’s value in that from a “business development” standpoint.”

Because of the structure of these two businesses, their records are organized by point of sale system and online versus physical locations. They both explained there is a business need to track total revenue; thus, they track e-Commerce revenue as easily identified by their business model.

When asked more about how e-Commerce records are identified in the data, participants shared that how e-Commerce purchases are fulfilled aligns with data tracking (i.e., if a business has physical locations that can fulfill online orders, they likely have a way to assign online orders to a location and provide location-level data). If a business fulfills online orders in another way (e.g., from a warehouse) then orders are likely to be grouped as “online” and not affiliated with a specific location. The businesses without physical store fronts where orders can be fulfilled from do not tie e-Commerce transactions to a physical location and therefore could not provide location-level data. Additionally, one participant expressed some confusion about how in-store online orders delivered to customers homes would be tracked in the data.

“I don’t know how sales would be attributed to the store in the situation where someone is ordering online while standing inside of the store. For the most part, our online sales are categorized as one specific location which is ‘online.’”

In addition to organizing e-Commerce records by location, or grouped together as “online” transactions, other businesses organize e-Commerce records by POS software. Three businesses indicated that they are able to identify their e-Commerce transactions by the POS software system used. Because only certain payment systems are used for what they consider e-Commerce transactions, they can easily pull out all the data from those systems. Two of these participants report for non-profit organizations, while one reports for a for-profit business. It is possible that non-profit organizations are more likely to have multiple point of sale systems that more easily translate to reporting e-Commerce activities, but further research is necessary.

“For example, our retail space, we do have, thinking about credit card transactions. You can go in and buy a t-shirt in the retail space by swiping your credit card. That transaction is processed through one software that we license, while someone who reserves an eco-tour, that transaction is processed through a different software system. So, it’s kind of siloed, so we’re able to know where these sources are coming from…when importing from the software it will say, for example, [Software A or Software B], I was looking for those specific words [in the records], and there were other ones where we had gotten contributions from wire transfer, so I included those [when reporting e-Commerce].”

Comparatively, it is also easy for businesses to track and report e-Commerce data when there are multiple revenue streams but online orders only use one software. In these cases, online transactions are typically flagged as “online” or “website” and can easily be separated in the data.

“If the sales are labelled as “website sale” then it is an e-Commerce [sale].”

“I know the categories that are e-Commerce because it is a progressive checkout, so when we run totals by a device or location that gives us that breakdown that we need. So, we can easily pinpoint what was received on site, whether it be at the front desk or if someone chose to go online and order admission tickets 17 days from now.”

In addition to reporting e-Commerce for locations, participants were asked about reporting by industry and for the overall company. All participants indicated that it would be easy to provide a company-wide estimate for their understanding of e-Commerce. Most participants could also report by industry, except one who indicated “Basing everything on an NAICS code is extremely difficult as that is not how we keep records.” This echoes findings from the concurrent AIES Main Study Debriefing Interviews that while reporting by industry is easy for most participants, a minority find it challenging because their records are not organized by industry.

Additionally, reporting e-Commerce activities becomes significantly more complicated when participants consider the current AIES question, especially the instructions to include BOPIS transactions and payment made in person. In these instances, most businesses would be unable to easily identify online reservations with payment in-store as e-Commerce, as mentioned in Section 3.3.2. Typically, these transactions would be assigned to the physical store where payment is made rather than online.

“I don’t know [if in-store payment for online orders would be considered e-Commerce]. That is a good question. The sale would then be attributed to the store. I don’t know how to think about that one if I’m being honest with you. If someone is buying it in the store and then shipping it then they are just shipping it out of the warehouse and it has nothing to do with an online sale. It seems more like a store sale.”

Another participant was asked whether they would be able to report instances when customers ordered online but paid in store.

“I think [it would be] difficult. I don’t have it broken out in the [general ledger]. It’s either mobile pay, or in-store. If we can pay in the store, then it’s probably going in the store sales and not the mobile pay. For mobile pay, I think the payment has to be done online.”

As outlined in Section 3.2.3, this participant was also confused about how to incorporate sales from in-store kiosks into her understanding of e-Commerce. When probed about whether kiosk sales would be separate in the data, she shared, “no, I think that’s included as part of store sales.” However, she did admit that she thought there could be a future business need to track in-store kiosks sales separately to better staff their locations.

Another finding for these business that track e-Commerce is that the scope of e-Commerce activities seems to correlate with the business need to track e-Commerce. Only one business that currently tracks their e-Commerce transactions indicated that e-Commerce comprises a small part of their revenue. The other nine businesses have either a large or medium share of their revenue as e-Commerce.

In summary, businesses that do track e-Commerce activities typically have a business model that maps well onto their conceptualization of e-Commerce. Most of these businesses include retail or restaurant establishments that can easily define online transactions as e-Commerce and have the records to support their definition. They typically have transaction-level data that are identified as originating from online, with payment made online. These businesses do not track online reservations for in-person payment and struggle to conceptualize these types of transactions as e-Commerce.

All revenue is e-Commerce (n=5)

Finally, the last category of businesses are those whose entire revenue can be considered e-Commerce. These five participants are tracking e-Commerce at the company level and can easily report one total figure for their business, the same as their report for total revenue. These businesses were from non-retail service industries such as non-profits, finanical, or professional services. Most participants could not provide more granular detail at a location or industry level. However, one participant said she could produce a report of e-Commerce revenue by location because their records are associated with the physical radio towers the station uses.

“Generally, we only look at it at the total. It would be a little more difficult to look at it by location because I would have to create a more granular report for that. I don’t think that kind of report exists right now. Maybe it would take 1-2 hours to generate a new report.”

This finding indicates that ability to report a more granular level of e-Commerce records for businesses where all revenue is e-Commerce may vary and be industry-specific. However, most of the businesses that indicated all their revenue is e-Commerce do not have physical locations for customers to purchase goods or services and thus would not be able to provide location-level data.

As described previously (see Section 3.3.3), one participant took a staunch position that “all revenue is revenue” and argued that all of the activities the business conducts are e-Commerce. However, when walking through the details of her organization, she did admit that some of the transactions do happen at brick-and-mortar locations. Regardless, without a business need to track e-Commerce, she was only ever willing to either report all revenue as e-Commerce or none at all.

“Depending on the specific revenue stream, it is either all online, for example joining, registering for a program, going to camp—100% online. If it is retail, 90% brick and mortar. If it is a donation, 50-50, maybe. Whether it is brick and mortar in person versus online, at this point in time is not something we are interested in tracking. So, revenue is revenue. We are not at a granular level where we need to know which sale was e-Commerce and which was brick and mortar.”

After reviewing the current e-Commerce question, one participant explained that if forced she would pick percentages at random to estimate the amount of e-Commerce the business engages in.

“If it was an absolute must, I would go through and just start picking percentages. I would probably do 80% e-Commerce and 20% brick and mortar and it would be totally random. I feel like you are asking me to provide an answer when we don’t have the infrastructure to do it properly so it might be made up.”

In the end, comprehension of e-Commerce and the question text are not the primary issues for this participant. Instead, the records for their business are not set up to capture e-Commerce, and she would rather report either all or none of the revenue than get embroiled in the minutiea of data requested by the AIES. This example illustrates that while the definition and comprehension of e-Commerce is important, some businesses will ultimately provide what data are easiest to access, regardless of whether they match all conditions given in the question.

2. Data availability impacts understanding of e-Commerce

Furthermore, the data businesses have available about e-Commerce can directly influence their comprehension of e-Commerce. When certain types of e-Commerce records are easy to access and report, those records become influential in defining what e-Commerce means. Participants report the data that are easily accessible, which helps solidify and reinforce their understanding of e-Commerce. One paritcipant, who has been discussed previously, believes that their business does not engage in e-Commerce because he does not include online reservations with in-store payment in his conceptualization of e-Commerce. After reviewing the current AIES question and discussing BOPIS orders, he shared the following:

“We aren’t currently taking credit cards online, someone can make a reservation but if they don’t show there is no consequence in that. I don’t know how we could calculate revenue based on our website. We can say if your revenue for a day was $10k, of that how much was generated from people who made reservations online, that would be very difficult to find.”

This example exemplifies the reflexive and intertwined nature of e-Commerce records and the definition of e-Commerce. This participant holds a very static understanding of e-Commerce, which does not include in-person payments. However, the records he accesses also do not make a distinction between in-store purchases and online orders with in-store payments. It is hard to determine how much the way the business records are organized influences his definition of e-Commerce versus how much his understanding of e-Commerce influences the way the records are organized. Business record organization is often not under the purview of the point of contact for the AIES and instead is influenced by internal stakeholders and historic precendence at each company.

Another participant, who believes their business engages in e-Commerce, shared how the availability of the records influenced his understanding of e-Commerce:

“I think when I read and was looking to complete the survey, my mind immediately went to the platforms I know we have, and not really diving into the nuance of, from the retail space this specific amount relates to ecotours that would have been reserved on this platform… I didn’t get into that level of it. I know that these ones over here are all electronically, so I just grabbed that data.”

This participant emphasized that he did not think extensively about what e-Commerce means but rather pulled the data that were easy to access and report that he knew would be considered e-Commerce. This example illustrates that some participants will read the question and report only the e-Commerce records they have easy access to without further considering what e-Commerce means. If the records they have easy access to match their definition of e-Commerce, they may not read the question instructions to further consider what to report. Another participant, who believes their business engages in e-Commerce, shared a similar strategy when reporting e-Commerce.

“I interpret this [question] as revenue. I was looking at the revenue that came from Shopify… I know there’s apps and stuff out there that the salespeople are using. In this case I applied what I knew… When I was doing the survey, I really knew that I could get the e-store [Shopify data], so that was my stop point. [These instructions don’t] surprise me, that is how I would define e-Commerce. It depends on whether you think of e-store as only type or if electronic could be anything e-Commerce, like email and apps [used by the salespeople].”

Because the Shopify data that she has easy access to matches her conceptualization of e-Commerce exactly, she chose to report only on that amount of revenue rather than consider other forms of potential e-Commerce, such as electronic sales made by their salespeople through email or other methods. When probed further about whether she would be able to identify e-Commerce transactions from the salespeople in the data currently, she explained that she could not. They have many methods of sales generated from their salespeople, including in-person and phone, that are entered directly into an application. She was unsure whether using the application alone should be considered e-Commerce. Regardless, the only distinction she could see in the data was between the Shopify sales and all other revenue. Again, this example shows that when participants have easy access to records that match their conceptualization of e-Commerce, they will report that data without considering e-Commerce further. This strategy becomes an issue because so many participants conceptualize e-Commerce in a different way than the full question instructions specify, leading to measurement error.

The two businesses that do not track e-Commerce because of no defined business need also illustrate how the definition of e-Commerce and the records a business keeps influence each other. One participant resolutely believed that online reservations with in-person payment should not be considered e-Commerce and the way his business’s records are currently organized also reflect this belief. Sales that are completed with in-person payment are affiliated with the location where the payment was made rather than flagged as an online sale. The other participant, who reports for a utility company, had similar issues with how e-Commerce was understood compared to the records available. This participant questioned whether online billing should be considered e-Commerce and shared that currently there is no way to identify bills paid electronically in the data. The lack of a business need to track e-Commerce influences how these two businesses organize their revenue records and thus may also play a role in how these two pariticipants conceptualize e-Commerce.

The reflexive nature of businesses’ e-Commerce records and conceputalization of e-Commerce when reporitng for the AIES is an important issue for us to consider. Measurement error is a serious concern because common conceptualization of e-Commerce is so misaligned with the AIES definition. Currently, businesses with e-Commerce activities (i.e., online transactions) that are clearly identified in the data can easily report e-Commerce. These are typically retailers and restaurants with take-away options. All other businesses struggle to match their understanding of e-Commerce to the parameters set by the AIES question, influenced by common perception of e-Commerce and the records the business keeps about e-Commerce.

3. Accessibility of e-Commerce records

In addition to understanding the record-keeping practices of businesses, we also wanted to better understand record accessibility. As outlined in the previous section, there are many people (23% of participants) serving as the point of contact for the AIES who do not have access to the e-Commerce records kept by their businesses. Accessibility of the records is closely tied to the record-keeping practices and the point of contact’s role at the business. Those in Human Resources and administrative roles are more likely to need to reach out to other people to gather e-Commerce records. Those who do not have access must collaborate with their colleagues to report e-Commerce data for the AIES.

As outlined in Section 3.4.1, businesses fall into one of five groups for their e-Commerce tracking behaviors. Whether the point of contact has access to the data typically aligns with these five groups (see Exhibit 3-6). A total of 10 participants (32%) reported that they could not access the e-Commerce data at their business. Businesses that have a point of contact who is not knowledgeable about e-Commerce records do not have access to these records (n=7). However, in addition there were three participants who were knowledgeable about the e-Commerce activities at their business who indicated they do not access the data and typically work with others to pull it. One of these businesses engages in e-Commerce but does not track it whereas the other two engage in e-Commerce and do track these activities. These two participants were knowledgeable about the records their business keeps but did not access the data themselves.

Exhibit 3-6. Accessibility of Records by Types of Businesses Tracking e-Commerce Activities

When the point of contacts cannot access e-Commerce data themselves, it is unclear how many colleagues they work with to gather the information they need. The high-level feedback from these participants were focused on the teams involved, such as the information technology department, without listing the number of people. When asked about their reporting process for the AIES overall, these participants listed many different departments that they must work with to complete the survey, such as finance, Human Resources, and point people at each location. One participant specified, “I can pull the data myself. For e-Commerce I can send it to someone else.” Four participants who cannot access e-Commerce records directly think others at the company may be a better point of contact than them and would save them the administrative effort of coordinating the response. One participant shared, “I think [with our] reporting and analytics [group] would be more helpful to get [e-Commerce] information.” Further research may be helpful to better understand the organizational burden that e-Commerce questions pose for these individuals.

Those who believe their business does not engage in e-Commerce (n=7) were not probed further on their access to e-Commerce records. However, most of these participants indicated that during their typical response to the AIES they access the data they need for the survey themselves. Only one of these participants reported that they had to work with others to respond to the survey.

The five businesses that report all their revenue is e-Commerce indicated that the point of contact could access the data. Only one participant reported that they have access to the data but typically do not pull the numbers themselves. These participants can report only e-Commerce revenue by the whole company, because their e-Commerce revenue equals their total revenue. The businesses included in this study where all their revenue is e-Commerce revenue do not have traditional physical store fronts where the products or services could be associated with in the records, so there is no location-level reporting feasible.

The last group are those who have access to the data and the business does track e-Commerce activities (n=8). For these eight businesses where the point of contact has access to the e-Commerce records, they typically can access all the levels of granularity available for this data. However, a few seem to have easier access to company-level reporting compared to finer granularity (e.g., transaction-level data).

“I can select and filter our records by location or other breakdowns to look at our records. The data from individual local stores are synced and imported into the accounting software for our use. And I am the ideal point of contact to get data about e-Commerce.”

“e-Commerce is just as accessible as other levels.”

However, one participant indicated that they could only access company-level data about e-Commerce. This was due to lack of records and reporting rather than a lack of access; this business does not affiliate e-Commerce records with locations because all online orders are fulfilled from a warehouse. Two participants who do not have access to the records were also knowledgeable about the effort it would take to access finer levels of granularity.

“It is possible to get data at different level. I’d talk to the Remarketing team and explain the reporting requirement, and then they would prepare the information. It would be a little bit more time consuming for more granular information.”

One of the primary research goals for this study was to better understand the accessibility of e-Commerce records for AIES respondents. However, there were a few challenges gathering in-depth feedback about accessibility. First, many of the participants (n=7) in this study did not believe that their business engages in e-Commerce. Second, although maintenance of the e-Commerce records was a key research question, the points of contact for the AIES are not typically involved with database management and decisions around record-keeping; they are unable to provide more detail about how the records are maintained. It is important to note that of the 24 businesses that do engage in e-Commerce, seven participants (29%) are not knowledgeable about e-Commerce records and cannot access the data. Future research should focus more on how these participants act as liaisons for their businesses and the definitional issues that may lead to measurement error when these participants do not understand e-Commerce.

5. Recommendations and Next Steps

Here, we highlight a few recommendations flowing from the findings across all the subsections of Section 3, including potential adjustments to the AIES question text and the need for further research.

1. Recommendations for the AIES question text

As summarized throughout Section 3.2 and Section 3.3, participant conceptualization of e-Commerce varies significantly from the current AIES question text, especially the provided instructions. We acknowledge that tracking e-Commerce serves an analytic goal for the Census Bureau. However, this research illustrates that the current AIES question suffers from the potential for significant measurement error because of varying respondent definitions of e-Commerce and lack of business records (see Section 3.4.2). We would encourage the Census Bureau to reevaluate the current definition of e-Commerce and confirm the analytic goals of measuring these activities to improve the validity of any statistics the Census Bureau is producing about e-Commerce. The recommendations outlined below reiterate this imperative to better align the Census Bureau question with popular conceptions of e-Commerce and existing business records. These recommendations should be considered in future research for additional testing, such as cognitive interviews.

• Recommendation 1: Consider reframing the official Census Bureau definition of e-Commerce to better match respondent conceptualization and business records to minimize measurement error. Participants struggled to consider online reservations with in-person payment as e-Commerce, and all businesses lacked the records to properly identify these transactions as e-Commerce. We recommend that the Census Bureau consider how important this e-Commerce activity is to the analytic goals served by the current question. This aspect of e-Commerce was identified as having the highest potential for measurement error because of disparate definitions of e-Commerce and lack of records. In addition, online reservations with online payment may also be omitted from reports of e-Commerce because of definitional issues rather than lack of records. Further collaboration with stakeholders such as data users and subject matter experts who are interested in e-Commerce to clarify which types of transactions should be included as e-Commerce will be helpful to reframe the definition of e-Commerce. For example, further questions could clarify whether the following types of transactions should be included:

  • Transactions mediated through salespeople (e.g., using an app or electronic tracking for sales)

  • Email negotiations for contracts

  • Property and equipment leasing

  • Online reservations for food services where the customer eats and pays in the establishment

  • Purchases made in person but paid with a credit card or virtual payment method

Because the design of the AIES seeks to present a single survey instrument applicable to businesses across all sectors of the economy, the effort to reframe the definition of e-Commerce should also consider the business activities of non-retail service industries and the non-service sectors. With a more balanced and comprehensive conceptualization, the Census Bureau will be able to consider more effective redesign of the e-Commerce question(s) that will address the issues of measurement error in a wholistic way. Furthermore, the reconceptualization effort may result in different emphases regarding the need to incorporate the tangible recommendations for the current AIES question listed below.

Based on the findings from the exploratory interviews, we also propose a few tangible recommendations regarding the current AIES question. It is worth noting that the subsequent tangible recommendations of additional industry-specific instructions and wording changes based on further cognitive testing may result in a nominal improvement of the current AIES question, although their implications for mitigating measurement error are secondary to the reframing effort.

• Recommendation 2: Consider presenting the e-Commerce question instructions prominently with the question text instead of hidden behind the expandable question mark. The question instructions for e-Commerce are key to respondent comprehension of the topic (see Section 3.3.2). When participants do not see the instructions, they operate with their own conceptualization of e-Commerce in tandem with the e-Commerce records that are easiest to pull to formulate their response (see Section 3.4). This outcome poses the risk of measurement error when respondents miss key aspects of the definition, such as monetary donations or BOPIS. Furthermore, even when respondents seek out further instructions about the e-Commerce question, they miss the expandable question mark. This tangible recommendation may be able to mitigate some potential measurement errors for the businesses that have accessible e-Commerce records with the data closely aligned with the Census Bureau’s definition of e-Commerce.

• Recommendation 3: Consider including additional industry-specific instructions to help non-retail industries map their business activities to the concept of e-Commerce. Participants from industries such as banks, utility companies, non-profit organizations and those that provide digital goods or in-person services struggled to understand e-Commerce in the context of their business activities, which do not align with the archetype of Amazon as e-Commerce (see Section 3.2.3). Providing industry-specific instructions should help these respondents better recognize e-Commerce activities and report for their businesses. Ideally, these instructions should not be hidden (per Recommendation 2) and may present based on NAICS code to only those businesses where they are applicable. Alternatively, reducing the reliance on retail-specific wording could also help increase the applicability of the question across industries.

• Recommendation 4: Consider removing the word “e-Commerce” from the question text entirely to avoid preconceptions about e-Commerce and force respondents to fully read the question text. This may help reduce measurement error but does not address the larger issues of measurement error outlined in the first three recommendations.

• Recommendation 5: Consider removing “shipments” and “receipts” from the first sentence of the question text. Participants use the words “sales” and “revenue” to understand what the question is asking about, and many do not find “shipments” or “receipts” to be applicable (see Section 3.3.1). Furthermore, “shipments” may cause confusion among those who define e-Commerce as only delivered goods, leading them to believe the question is only interested in goods shipped directly to consumers.

• Recommendation 6: Consider listing out exclusions for what types of transactions should not be included as e-Commerce. Participants are accustomed to seeing both inclusions and exclusions in question instructions, and some would prefer to see exclusions with the e-Commerce question (see Section 3.3.2).

• Recommendation 7: Consider updating the phrase “Electronic Data Interchange (EDI)” to a more modern wording. This phrase is seen as dated and confusing for participants who are not sure whether their systems are considered an EDI or not (see Section 3.3.2).

• Recommendation 8: Consider omitting the reference to “m-commerce.” Participants are unfamiliar with this phrase because online transactions happen seamlessly on both mobile devices and computers. The phrase can be confusing and cause participants to wonder whether orders placed through mobile phone calls should be considered e-Commerce (see Section 3.3.2).

• Recommendation 9: Consider future research on the phrase “remitting payment” in the context of e-Commerce. Currently, participants consider online orders where payment is made online as the exemplar of e-Commerce. However, the Census Bureau definition includes any transactions in which services or products are reserved or ordered online and then paid for in person. In the current study, only one participant was able to reconceptualize his understanding of e-Commerce activities by considering the order separate from how payment was remitted (see Section 3.3.2). Therefore, using this type of language in the e-Commerce question may help other respondents also distinguish these two actions (i.e., order and payment).

2. Recommendations for future research

In addition to the changes recommended to the question text, there is additional research needed as the e-Commerce question is updated and changed to better understand respondent.

• Recommendation 10: Consider additional usability testing of the current AIES survey format, which hides additional instructions behind a clickable question mark. This research illustrated that these instructions are not prominent for respondents and sometimes missed, even when respondents are seeking out additional instructions (see Section 3.3.4).

• Recommendation 11: Consider additional research with businesses that do not report e-Commerce revenue on the current AIES question but do have activities that could be considered e-Commerce. As illustrated by the current research, even those who provided a response that indicated they do have e-Commerce revenue sometimes do not believe they have any e-Commerce activities. Further research should target those who report $0 e-Commerce revenue but are believed to have e-Commerce activities to learn more about what leads them to report $0. Research questions should focus both on comprehension of the question and record-keeping.

• Recommendation 12: Consider additional research about how much organizational burden unknowledgeable point of contacts may face when answering e-Commerce questions. The current research did not capture a full picture of how many other people these participants have to work with to gather e-Commerce data to report. Furthermore, these participants can encounter some unique challenges with understanding e-Commerce and identifying the correct point person for reporting the data.

• Recommendation 13: Consider additional research on how important respondents believe a customer-facing website is for e-Commerce transactions. Several participants mentioned that they did not consider some transactions e-Commerce because customers were not interfacing directly with a website and entering the order themselves (see Section 3.2.1 and Section 3.2.3). It is unclear how important this concept is for e-Commerce. Future research could help us better understand whether this is a key definitional aspect for respondents and how to include it in the AIES question.

4. Concluding Remarks

The findings from the exploratory interviews revealed a complex relationship between the definition of e-Commerce and businesses’ record-keeping practices and how such a relationship may condition the reporting of e-Commerce activities. The record-keeping practices are highly dependent on the historical precedence of business operations, which may narrowly limit the scope of e-Commerce activities for recording-keeping or determine the need to track e-Commerce activities for operational support. Similarly, respondent definitions of e-Commerce activities may be driven by record-keeping practices that reflect business operations regardless of the roles of digital technologies and only consider the available records as the information they need to report.

Because respondents often consider online retailers like Amazon as the archetype of e-Commerce, such an incomplete definition may restrict respondents’ ability to retrieve and curate the accurate information from the available business records. Perhaps more interestingly, when there is a mismatch between respondents’ “archetype” definition of e-Commerce activities and the information sought by Census Bureau, respondents seem to be prone to misreport because of the potentially irreconcilable mismatch.

Taken together, the findings of the exploratory interviews showed some examples from a small number of industries about the high variation in the definition and record-keeping practices of e-Commerce activities from a business perspective. More research is needed to further understand the dynamics that influence the reporting of e-Commerce activities.

The findings and recommendations presented here will be used to consider the research agenda and the scope of the next iteration of exploratory research of e-Commerce activities, update the legacy questions used in the current AIES, and potentially develop a new set of questions that seek to better measure e-Commerce activities.

Appendix A. Exploratory Interview Protocol

2024 AIES E-Commerce Exploratory Interviewing Protocol

Participant ID
Date
Time
Interviewer
Multiple Industries?
Manufacturer?

Purpose:

Researchers in the Census Bureau’s Economic Statistical Methods Division (ESMD) and Economy-Wide Statistics Division (EWD), and with RTI International, will conduct exploratory interviewing to investigate the record-keeping practices and data accessibility of e-Commerce economic activity at respondent businesses.

These interviews will cover three main topics: record-keeping practices regarding e-Commerce activity; data accessibility of records related to e-Commerce; and current e-Commerce reporting practices on the AIES. Not all questions in this protocol will pertain to all participants, and some modules are optional depending on the length of the interview and firm characteristics. Numbered questions are priorities, and bulleted questions underneath are optional probes.

Research Questions: The research will be guided by the following research questions:

1. What are the current e-Commerce record-keeping practices of businesses?

• What is included or excluded in these records?

• What are the differences in record keeping by firm industry(ies)?

• What are the differences in record keeping by firm size and complexity?

• At what level of granularity are these records kept?

2. How accessible are records of e-Commerce activity?

• How easy or challenging is accessing this information?

• How many people at the business are involved in maintenance and pulling of e-Commerce records for reporting?

• Are there levels of granularity that are more or less accessible for respondents (e.g., establishment-level compared to industry- or company-wide level)?

3. How do current e-Commerce record keeping practices inform current reporting practices on the AIES?

• What are respondents currently including (and excluding) from reporting e-Commerce for AIES?

• How much data manipulation (aggregation, allocation, estimation, and others) do respondents engage in to report e-Commerce for AIES?

• What are the factors that determine decision to report e-Commerce on the AIES?

Informed Consent: Respondents will be asked to complete a consent form electronically before the time of the interview.

Materials Needed:

• Electronically signed consent form

• Respondent recruitment and firm information, including: Contact info; establishments in manufacturing; single/multi-units; number of establishments; number of industries

• Current AIES harmonized E-Commerce question text

Method: We will conduct the interviews by Microsoft Teams (or phone).

Expected length of interview: 1 hour (60 minutes) maximum

General probes that may be used throughout the interview:

• Were these data easy to access?

• What else can you tell me about this?

• Can you tell me more about that?

• How confident are you in that response?

• What looked unclear or is confusing here?

Introduction (5 minutes)

If necessary: You should have received a link to a consent form from our recruiter via email. Did you have a chance to review and sign that yet?

[IF YES] Did you have any questions about any of the information presented there?

[IF NO] Please open up the link now and review the form, then sign it. Take your time and let me know if you have any questions.

Thank you so much for agreeing to talk with me today!

I am working with the Census Bureau to get feedback about the Annual Integrated Economic Survey, or AIES, and review some questions that we are developing for future use on the survey.

My job is to improve questions on our economic surveys, and particularly on the Annual Integrated Economic Survey. I didn’t write the survey questions, so you do not have to hold back when telling me your feedback. Please be candid and frank in your responses. Our interview is being conducted under the authority of Title 13, which means that your responses are confidential, and neither your name nor the name or identifying information about your company will be included in any of our findings.

Do you have any questions before we get started?

[IF OBSERVER:] I also have an observer here on the call today. They are here primarily to observe my performance.

[IF PARTICIPANT DOES NOT WANT OBSERVER TO JOIN, ASK OBSERVER TO DROP OFF THE CALL.]

I’d like to record our session today so that when I go to analyze the results of these interviews, I can use the recording to pick up on anything I may have missed in my notes. Do I have your permission to record our session today?

[Turn on recording]

[State respondent ID number and date, repeat consent on recording: Do you agree to participate in the interview today? Is it okay if I record our conversation?]

Topic 1: Definitions and Record-Keeping Practices

Module 1: Warm up (10 minutes)

Universe: All participants

Let’s get started with some general questions about your job and Census Bureau surveys in general.

[INTERVIEWER NOTES: The goal for this section of questions is to understand the respondent’s role in the business and ask overall about their experience answering Census Bureau surveys. This section should help you establish rapport with the respondent. Use this section to learn about how the business reports for Census surveys, including the respondent’s role and other people they may work with to report data.]

1. Tell me a little bit about your business. What types of goods or services does this business provide?

   • How does your business make sales of these goods/services?

2. [IF NEEDED:] Does your business have physical locations? Where are these locations, that is multiple states? Nationwide?

3. What is your role in the company? How long have you been in this role?

4. What is your role in the process for responding to Census Bureau surveys in general?

   • How do you find the data you need to answer Census Bureau surveys?

     1. Do you work with anyone else to get the data you need?

     2. Do you access any reports to get the data you need?

   • How easy or difficult is it for you to find all the data you need for Census Bureau surveys? Why is that?

   • How easy or difficult is it for you to enter the data into the survey once you have the answers you need? Why is that?

5. Do you have any other feedback about your experience responding to Census Bureau surveys before we move on?

6. I also want to learn more about how the data you use to report for Census Bureau surveys is organized. What software system do you use? How are the data organized within the software system? For example, by location, by revenue stream, by product type, or something else? [INTERVIEWER NOTE: We want to better understand the level of granularity for their records and what kind of information they can easily access. Respondents may report data in different software or systems (i.e., a “patchwork of systems.” Do not get too bogged down in the details of the data, we are most interested in revenue and accounting data.]

Module 2: Definitions (5 minutes)

Universe: All participants

Thanks for that information. Now we’re going to talk about a specific topic that you may or may not have reported about before on Census Bureau surveys. We’re focused on this topic because businesses have many different ways of organizing their records and we want to learn more about what your business does.

[INTERVIEWER NOTES: The goal for this section of questions is to understand the respondent’s definition of e-commerce. Previous research has shown that defining e-commerce is very difficult. We are looking for what respondent’s think e-commerce is. They may mention things about the ordering and delivery process, the monetary transactions, the nature of the products/services provided, and the actors involved (such as digital intermediaries). Do not provide definitions to the respondent. Leave this very open-ended. This is not the focus of our research. All respondents sampled have reported e-commerce data but may not be able to define it.]

1. Let’s think about the word “e-Commerce” – what is e-Commerce, in your own words?

   • How can you tell if a transaction is e-Commerce or not?

   • What does this include? What does it exclude?

[INTERVIEWER NOTES: We’re looking for things like the types of products or services provided and how the transaction takes place.]

2. Does your business engage in e-Commerce?

[INTERVIEWER NOTES: All respondents sampled have reported e-commerce data but may not be able to define it or it may not be clear that they have reported e-commerce data.]

• [IF YES:] What are some examples of ways your business engages in e-Commerce?

• [IF NO:] Could you describe a bit more about the typical transaction process for your business?

• What were you thinking about when you said that your business [does / does not] engage in e-Commerce?

• What did you include or exclude when you said that your business [does / does not] engage in e-Commerce?

[INTERVIEWER INSTRUCTION: If respondent indicates that their firm is not engaging in e-Commerce, skip to Module 5.]

Module 3: Record-Keeping Practices (20 minutes)

Universe: All participants that indicated e-Commerce activities

[INTERVIEWER NOTES: All respondents sampled have reported e-commerce data but may not be able to define it or it may not be clear that they have reported e-commerce data. Use your best judgement to proceed with the questions in this section. This is the most important section of the interview. We really want to know more about how e-commerce data is tracked and stored. Previous research shows that many businesses do not have access to e-commerce data because tracking it is not relevant to their business needs.]

1. [IF RESPONDENT REPORTS ENGAGING IN E-COMMERCE:] Tell me about how your company tracks your e-Commerce activities – what is included in this tracking?

[INTERVIEWER NOTES: We want to know more about the break down for sales records and how they are recording sales. It’s common for businesses to break down their sales by revenue streams, which include broad categories like revenue from goods sales/service fees, interest revenue, and rent revenue. However, the revenue from goods/services may not be broken out by mode of sale (e.g., online versus in store).]

• What types of activities are included in your e-Commerce tracking? I’d like to learn more about these e-Commerce activities of your business, what sources of activities are included in your e-Commerce tracking?

• What products or services are included in your company’s e-Commerce records? What is excluded?

• With what frequency are e-Commerce records updated?

• How confident are you that your business is consistently tracking all of your e-commerce activities?

• How important is tracking e-Commerce activities for your business? Why is tracking e-Commerce activities [important/not important] for your business?

1. Are your e-Commerce records kept separately from other types of records?

   • [IF YES:] how are they maintained separately? Can you tell me more about that?

   • [IF NO:] how can you tell e-Commerce from other types of records? Can you tell me more about that?

2. Are your e-Commerce records kept differently for different parts of your company?

   • That is, do you track this information differently for different parts of your company? [IF YES:] how? [INTERVIEWER NOTES: Probe as needed about potential differences in the types of information tracked, such as differences by location, industry type, or type of sale.]

   • How are these differences reflected in your records?

3. At what level are you tracking e-Commerce at your company, that is, what is the lowest level of granularity you can access?

   • For example, can you provide e-Commerce revenue for each location at your company?

   • Can you provide e-Commerce revenue for groups of locations that make or do the same thing?

   • Can you provide e-Commerce revenue for your company as a whole?

   • [IF BUSINESS HAS PHYSICAL LOCATIONS:] How do you determine “where” to track e-Commerce activities? That is, if an order is placed online, how is it recorded in your records relative to your physical locations?

Topic 2: Data Accessibility

Module 4: Accessibility of E-Commerce Records (10 minutes)

Universe: All participants that indicated e-Commerce activities

[INTERVIEWER NOTES: All respondents sampled have reported e-commerce data but may not be able to define it or it may not be clear that they have reported e-commerce data. Use your best judgement to proceed with the questions in this section. This is the most important section of the interview. We really want to know more about how e-commerce data is tracked and stored. Previous research shows that many businesses do not have access to e-commerce data because tracking it is not relevant to their business needs. Use the information you learned at the start of the interview to inform your probing in this section.]

1. Now that I have a better understanding of what e-Commerce activities you are tracking at your company, I’m wondering about how easy or difficult it is to access those records. Will you please describe for me how you go about accessing your e-Commerce records?

2. Can you access the databases where these records are kept on your own, or do you have to rely on others within your company? How many other people are involved?

3. Are there levels of granularity that are more accessible than others for e-Commerce?

   • For example, is it easier to access e-Commerce for each of the locations at your company, for groups of locations that do or make the same things, or for your company overall?

   • If you wanted data at the location level, would you have a central location to get those data or would you need to reach out to each location, or something else? Please describe the process of getting location-level e-Commerce data if available.

   • Who is the ideal point of contact to get data about e-Commerce for your company overall? How about for each location at your company?

Topic 3: Reporting E-Commerce

Module 5: E-Commerce Reporting Practices on the AIES (15 minutes)

Universe: All participants

Now I’d like to review a survey question with you about reporting e-Commerce data. There are no right or wrong answers, I just want to learn more about how you understand this question to make sure it’s working well for everyone. [INTERVIEWER INSTRUCTION: share screen to respondent and confirm they can view your screen. If possible, share only the window with the e-commerce question. Share current AIES e-Commerce question ONLY, that is the first page of the PDF].

[IF BUSINESS HAS MULTIPLE INDUSTRIES:] This question is presented for each industry that the business has so that data can be collected at the industry-level for a given company.

[IF BUSINESS INCLUDES MANUFACTURING:] This question is presented for each manufacturing location so that data can be collected about each location.

1. Take a minute to read over this question. Can you tell me in your own words what we are asking here?

2. Thank you for that feedback. Now I want to show you some additional instructions that are available when answering this question by clicking on an icon with a question mark next to the question text. [INTERVIEWER INSTRUCTION: Go to second page of PDF to share current AIES e-Commerce question with instructions].

   • Now that you’ve seen the instructions, do they change what you think we are asking here, or does this match what you were thinking about?

   • What surprises you, if anything, about this question or the additional instructional information about this question?

   • Have you seen a question like this before?

     1. [IF YES:] Do you remember whether you viewed the additional instructions or not? Why?

        1. [IF DOES NOT REMEMBER WHETHER VIEWED INSTRUCTIONS:] Do you typically read additional instructions when you respond to Census Bureau surveys? Why?

   • [IF PARTICIPANT INDICATED THAT THEIR FIRM DOES NOT ENGAGE IN E-COMMERCE OR DID NOT KNOW WHAT E-COMMERCE IS, ASK:] Does seeing the question change whether or not your company engages in e-Commerce? Why or why not?

3. How would you go about answering this question? What would be your first step? And then what would your next step be? [INTERVIEWER NOTE: probe as needed to make sure respondents are thinking about reporting this at an industry-level for multiple industries and at a location level for each manufacturing location.]

   • Would you need to reach out to anyone else in your company? [IF YES:] Who?

   • How much time do you think it would take you to answer this question, and why?

4. How easy or difficult would it be to answer this question based on the records you have available?

   • How close to the way you keep your records is this question?

   • Would you need to add, allocate, estimate, or otherwise manipulate your data to fit the question?

   • How easy or difficult would these calculations be? How much time do you think it would take you?

   • Are there things we are including or excluding that do not match how you keep your records?

     1. [IF THERE ARE MISMATCHES:] How would you reconcile the data to report an answer if this question was on a Census Bureau survey?

5. How likely are you to provide a response to this question?

   • Why did you answer that way? What were you thinking about?

   • How can we make it easier for you to report your e-Commerce activities?

   • What do you think would happen if you purposefully left the question blank?

6. Are there internal reports you run about your e-Commerce? Who are the stakeholders for these internal reports? How do they use the internal reports?

7. Are there other instances of external reporting – like for taxes, or for shareholders, or for regulatory reasons – that you are reporting e-Commerce?

   • [IF YES:] Tell me about them – is it easier or more difficult than reporting to this question and why? Probably the same.

[INTERVIEWER NOTE: If respondent previously reported they do not engage in e-commerce but then changes their response after reviewing the question, go back and administer Module 3 and Module 4].

Wrap up/Debriefing (5 minutes)

Universe: All participants

1. That’s all the questions I have for you today! Is there anything else we did not talk about today that we should be thinking about? Do you have any other comments, questions, or suggestions for us?

[IF OBSERVER AND TIME LEFT:] I just want to check in with our observer and just see if they have any follow up questions for you before we close.

Thank you so much for your time today.

Appendix B. Current AIES e-Commerce Question

Revenue from Electronic Sources

Electronic Commerce (e-commerce): Part of total revenue

Of the total amount of Sales, Shipments, Receipts, or Revenue reported, what amount was from goods, services, or manufactured products that were ordered or whose movement was controlled or coordinated electronically?

Please provide an estimate if exact figures are not available.

2023 (report in thousands)

Revenue from Electronic Sources

Electronic Commerce (e-commerce): Part of total revenue

Of the total amount of Sales, Shipments, Receipts, or Revenue reported, what amount was from goods, services, or manufactured products that were ordered or whose movement was controlled or coordinated electronically?

Please provide an estimate if exact figures are not available.

2023 (report in thousands)

Electronic Commerce (e-commerce) or e-shipments:

The sale of goods, services, or manufactured products where the buyer/customer places an order, or the price and terms of the sale are negotiated, over the Internet, extranet, email, mobile devices (m-commerce), an Electronic Data Interchange (EDI) network, private networks, dedicated lines, or other comparable online systems/applications.

Payment may or may not be made online (e.g., order online but pay at a location in person).

Include:

• Revenues from customers entering orders directly on your own and/or third party websites or mobile applications

• Shipments to other domestic plants of your own company for further manufacture, assembly, or fabrication

• Buy or reserve online, pick up in store (BOPIS)

• Buy or reserve online with home/business delivery

• Buy online with curbside pickup

• Buy online with kiosk/locker/tower pickup

• Monetary donations received online as revenue from electronic sources (for tax-exempt firms)

1 For more information, please visit https://www.census.gov/programs-surveys/aies.html and https://www.census.gov/programs-surveys/aies/technical-documentation/methodology.html

2 Hughes, K. A. (2004). Comparing pretesting methods: Cognitive interviews, respondent debriefing, and behavior coding (Survey Methodology #2004-02). Statistical Research Division, U.S. Census Bureau.

3 Fayyaz, S. (2018). A review on measuring digital trade & e-Commerce as new economic statistics products. Statistika: Statistics and Economy Journal, 99.

4 Hsieh, Y. P., Stenger, R. A., Blackburn, K. M., Timbrook, J., & Ellis, C. (2022). Cognitive testing support project: Final report on development and cognitive testing of Emergency Economic Information Collection (EEIC) question bank. U.S. Census Bureau.

5 These subjective distinctions were made based on the information provided by participants during the interview about the amount of e-Commerce their business conducts, including details about the type of sales and how important the sales are to the business, and how these participants described the volume (such as the quantifiers they used during the discussion).