ABS Finance and Management Practices Cognitive Testing Results

F. ABS Finance and Management Practices Module Cog Testing F&R.pdf

Annual Business Survey

ABS Finance and Management Practices Cognitive Testing Results

OMB: 0607-1004

Document [pdf]
Download: pdf | pdf
Findings and Recommendations from
Cognitive Testing for the Annual Business
Survey – Finance and Management
Practices Module
Prepared for:
Patrice Hall, Economic Reimbursable Surveys Division (ERD)
Aneta Erdie, ERD
Audrey Kindlon, National Center for Science and Engineering Statistics (NCSES)
John Jankowski, NCSES

Prepared by:
Krysten Mesner, Data Collection Methodology and Research Branch (DCMRB)
Aryn Hernandez, DCMRB

Economic Directorate
U.S. Census Bureau

October 26, 2020

1
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Contents
Research Objectives ................................................................................................................................ 4
Research Methodology............................................................................................................................ 5
Participants ............................................................................................................................................. 6
Findings and Recommendations .............................................................................................................. 6
General Findings and Recommendations ................................................................................................. 6
Finding #1: Estimated time to complete averaged approximately 30 minutes. ..................................... 6
Question Specific Findings and Recommendations .................................................................................. 6
A.

Financing ..................................................................................................................................... 6

Finding #2: Capital funding .................................................................................................................. 6
Finding #3: Amount of capital needed to start or initially acquire the business .................................... 6
Finding #4: Funding from owner(s) ...................................................................................................... 7
Finding #5: Profitability........................................................................................................................ 7
Finding #6: Negative impact on profitability......................................................................................... 7
Finding #7: Operational changes as a result of coronavirus pandemic - #6 ........................................... 8
Finding #8: Operational changes as a result of coronavirus pandemic - #7 ........................................... 8
Finding #9: Operational changes as a result of coronavirus pandemic - #8 ........................................... 8
Finding #10: Operational changes as a result of coronavirus pandemic - #9 ......................................... 8
Finding #11: Operational changes as a result of coronavirus pandemic - #10 ....................................... 9
Finding #12: Operational changes as a result of coronavirus pandemic - #11 ....................................... 9
Finding #13: Operational changes as a result of coronavirus pandemic - #12 ....................................... 9
Finding #14: Operational changes as a result of coronavirus pandemic - #13 ....................................... 9
Finding #15: Working from home ........................................................................................................ 9
Finding #16: Percent of employees working from home ...................................................................... 9
Finding #17: Factors affecting work from home ................................................................................. 10
Finding #18: New credit applications ................................................................................................. 10
Finding #19: Credit sources................................................................................................................ 10
Finding #20: New credit applications amount .................................................................................... 10
Finding #21: Business financing uses.................................................................................................. 11
B.

Management Practices .............................................................................................................. 11

Finding #22: Managing service or production problems ..................................................................... 11
Finding #23: Number of key performance indicators.......................................................................... 11
Finding #24: Frequency of key performance indicators - #3 ............................................................... 11
2
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Finding #25: Frequency of key performance indicators - #4 ............................................................... 11
Finding #26: Business targets - #5 ...................................................................................................... 12
Finding #27: Business targets - #6 ...................................................................................................... 12
Finding #28: Employee promotion ..................................................................................................... 12
Finding #29: Under-performing employee ......................................................................................... 12
Finding #30: Employee training expenditures .................................................................................... 12
Finding #31: Employee training participation - #10 ............................................................................ 13
Finding #32: Employee training participation - #11 ............................................................................ 13
Finding #33: Employee training participation - #12 ............................................................................ 13
Appendix A: Draft Questions, Round 1 ................................................................................................... 14
PROFITABILITY ...................................................................................................................................... 16
NEGATIVE IMPACT ON PROFITABILITY.................................................................................................. 17
SMALL BUSINESS CREDIT ...................................................................................................................... 20
NEW CREDIT APPLICATIONS ................................................................................................................. 20
CREDIT SOURCES................................................................................................................................... 20
Appendix B: Interview Protocol, Round 1............................................................................................... 25
Cognitive Interviewing Guide: Draft Content for 2021 ABS (4th Year).................................................. 25
Thanks for providing some background information. Now let’s look at the survey questions. ........... 25
Direct participant to look at draft content. ....................................................................................... 25
Appendix C: Draft Questions, Round 2 ................................................................................................... 33
PROFITABILITY ...................................................................................................................................... 34
NEGATIVE IMPACT ON PROFITABILITY.................................................................................................. 35
SMALL BUSINESS CREDIT ...................................................................................................................... 37
NEW CREDIT APPLICATIONS ................................................................................................................. 37
CREDIT SOURCES................................................................................................................................... 38
Appendix D: Interview Protocol, Round 2 .............................................................................................. 42
Cognitive Interviewing Guide: Draft Content for 2021 ABS (4 th Year) ............................................... 42
Thanks for providing some background information. Now let’s look at the survey questions. ........... 42
Direct participant to look at draft content. ....................................................................................... 42
Appendix D: Interview Protocol, Round 2………………………………………………………………………………………………66

3
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Research Objectives
The Annual Business Survey (ABS) asks questions about the business owner(s) as well as various
characteristics about the business. The 2021 ABS cognitive testing included new questions in the finance
and management practices modules. These proposed questions focused on:
•
•

Financing – asks about how the business was originally financed, current profitability, current
debt, and sources of credit.
Management Practices – asks about how service and production issues are handled, key
performance indicators tracked, business targets, and training. This module also included
questions about employee ability to work from home and about the impact of COVID-19 on
sales and financial assistance.

These draft questions were discussed using cognitive interviewing. The objectives of these interviews
were to learn how respondents:
•
•
•
•

understand and interpret the data requests;
understand the terminology used throughout the questions;
keep records related to the data items; and,
perceive the burden that would be associated with answering the questions.

The round one finance and management practices module draft can be found in appendix A. The round
one protocol can be found in appendix B. The round two finance and management practices module
draft can be found in appendix C. the round two protocol can be found in appendix D.

Note: The two rounds of testing were conducted in close proximity, time-wise, from each other.
Between the rounds of testing, some changes were made to the draft materials. This report will attempt
to summarize findings from all interviews, but the primary focus will be on the findings from Round 2 of
the interviews, and provides recommendations based on those draft questions.

4
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Research Methodology
Between June and September 2020, Aryn Hernandez and Krysten Mesner from the U.S. Census Bureau’s
Data Collection and Methodology Research Branch (DCRMB) conducted 17 cognitive interviews over
two rounds of testing, regarding proposed new questions on finance and management practices.
Participants in the interviews were recruited from a list provided by the Federal Reserve Banks of
Atlanta and Cleveland.
Cognitive interviews were the testing methodology used for this project. Cognitive interviews are used
to, ‘(a) understand the thought processes used to answer survey items, and (b) to use this knowledge to
find better ways of constructing, formulating, and asking survey questions’ (Forsyth and Lessler, 1991).1
Cognitive interviews traditionally focus on the four steps of Tourangeau’s (1984) cognitive response
model: comprehension, retrieval, judgment, and communication/reporting.2 Comprehension refers to
the respondent’s interpretation and understanding of the question’s language, structure, and grammar.
In order to answer the question, a respondent must understand what information is being requested on
the survey. Retrieval is the step where relevant information is obtained, either from records or from
memory. The next step, judgment, describes the respondent’s evaluation of the completeness or
relevance of the data obtained. It is here that estimates are made based on partial or incomplete data.
The last step, communication or reporting, deals with mapping the response to the answer space
provided and possibly altering the answer.
While Tourangeau’s model is suitable for household and social surveys, the establishment survey setting
presents additional factors that must be considered. First, instead of or in addition to a reliance on
memory, establishment surveys rely heavily on records and the information contained within them.
Second, organizations tend to have distributed knowledge. Some people are experts in one type of
information, while others keep information about something else. Third, competing priorities, both for
the organization and the individual(s) completing the questionnaire, mean that the survey sometimes
does not receive the amount of attention that researchers and data collectors would like. Finally,
organizations regularly authorize only a few individuals to release data. If the data provider is not
authorized to release the data, an additional step must be added to the response process. Tourangeau’s
model was expanded by Sudman et al (2000) to account for these factors. 3

1

Forsyth, B.H. and Lessler, J.T. (1991). ‘Cognitive Laboratory Methods: A Taxonomy.’ In Measurement Errors in Surveys, P.P. Biemer, R.M.
Groves, L.E. Lyberg, N.A. Mathiowitz, S. Sudman (eds). New York: John Wiley & Sons, Inc.
2

Tourangeau, R. (1984). ‘Cognitive Sciences and Survey Methods.’ In Cognitive Aspects of Survey Methodology, T.B. Jabine, M.L. Straf, J.M.
Tanur, and R. Tourangeau (eds). Washington, DC: National Academy Press.
3

Sudman, S., Willimack, D.K., Nichols, E., and Mesenbourg, T.L. (2000). ‘Exploratory Research at the U.S. Census Bureau on the Survey Response
Process in Large Companies.’ Paper prepared for presentation at the Second International Conference on Establishment Surveys, Buffalo, NY.

5
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Participants
Participants included contacts established by the Federal Reserve banks of Atlanta and Cleveland.
Businesses recruited covered a variety of industries that provide services and goods. Participants were
higher level employees with roles such as owner, president, CFO, CEO, etc. and, given their position in
the company, they were able to answer most of the questions that were tested.
Table 1. Count of businesses by type and round
Count

Round 1

Round 2

Total

Services

8

7

14

Goods

1

1

3

9

8

17

Total

Findings and Recommendations
General Findings and Recommendations
Finding #1: Estimated time to complete averaged approximately 30 minutes.
After reviewing the draft module, most respondents gave a time estimate to complete the module
between 10 minutes and 45 minutes.

Question Specific Findings and Recommendations
A. Financing
Finding #2: Capital funding
Most participants were able to answer this question. However, as previously stated, most of the
participants interviewed were much higher up than our typical respondent. If the company was very old,
this information may not be available to some of our respondents. A few participants mentioned that
sources such as issuance of stock or other public debt were missing.
Recommendation(s):
•

Consider adding ‘issuance of stock or debt,’ if appropriate.

Finding #3: Amount of capital needed to start or initially acquire the business
The majority of participants could answer this question or felt that they could answer this question with
sufficient research. These participants also felt that they could provide an estimate of the dollar
6
DRB Clearance Number: CBDRB-FY21-ESMD001-005

amount. However, typical respondents to the ABS may not have access to this information, especially in
larger and/or older companies. In addition, some larger companies felt that these ranges were geared
towards much smaller companies.
Recommendation(s):
•
•

Change answer choices into a dollar amount estimate.
Also, consider including a check box for ‘Don’t know.’

Finding #4: Funding from owner(s)
Most participants felt that they could answer or would be able to answer this question with sufficient
research. These participants also felt that they could provide an estimate of the dollar amount.
However, typical respondents to the ABS may not have access to this information, especially in larger
companies. In addition, some larger companies felt that these ranges were geared towards much
smaller companies.
Recommendation(s):
•
•

Change answer choices into a dollar amount estimate.
Also, include a check box for ‘Don’t know.’

Finding #5: Profitability
In testing this question performed well.
Recommendation(s):
•

No recommendations needed.

Finding #6: Negative impact on profitability
Participants were all able to answer this question. However, some participants felt that some factors
were missing from the list:
•
•
•
•
•

The cost of goods
Availability of supply
Market conditions
Competition
Trade policies

Recommendation(s):
•

Consider adding any of the suggested categories, if appropriate. Also, consider adding an ‘other
– specify’ category.

7
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Finding #7: Operational changes as a result of coronavirus pandemic - #6
Most participants were able to answer this question. Most participants measured significant vs.
somewhat in percentages. Many stated that ‘somewhat’ was 10% or less, while ‘significant’ was
anything over that.
Recommendation(s):
•

No recommendations needed.

Finding #8: Operational changes as a result of coronavirus pandemic - #7
Most participants were able to answer this question. However, a few participants stated that they
received funds from the CARES Act Funds, HEERF Relief Funds, and or deferred FICA. These participants
did not “request” this assistance. These participants stated that they would still answer ‘yes’ to this
question. One participant also mentioned assistance from private financial institutions such as
deferments.
Recommendation(s):
•

If assistance such as the CARES act should not be included, add an exclude instruction. If the
CARES act should be included, add that to the example in the includes.

Finding #9: Operational changes as a result of coronavirus pandemic - #8
Participants had no issues providing dollar amounts for this question. One participant didn’t understand
the difference between questions 8 and 9.
Recommendation(s):
•
•

Change ‘sought’ to ‘requested’ and bold this term to help differentiate this question from the
following.
In question 10, we use the phrase ‘government-sponsored coronavirus pandemic financial
assistance,’ we can reiterate this in questions 8 and 9 to be consistent.

Finding #10: Operational changes as a result of coronavirus pandemic - #9
Participants had no issues providing dollar amounts for this question. One participant didn’t understand
the difference between questions 8 and 9.
Recommendation(s):
•
•

Bold ‘received’ to help differentiate this question from the previous.
In question 10, we use the phrase ‘government-sponsored coronavirus pandemic financial
assistance,’ we can reiterate this in questions 8 and 9 to be consistent.

8
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Finding #11: Operational changes as a result of coronavirus pandemic - #10
Many participants were unable to answer this question with confidence. Many did not know how much
they would have to pay back because they had not been informed yet. While these participants believed
they would know by 2021, they could not be sure.
Recommendation(s):
•

Add a response option for Don’t know.

Finding #12: Operational changes as a result of coronavirus pandemic - #11
In testing this question performed well.
Recommendation(s):
•

No recommendations needed.

Finding #13: Operational changes as a result of coronavirus pandemic - #12
In testing this question performed well.
Recommendation(s):
•

No recommendations needed.

Finding #14: Operational changes as a result of coronavirus pandemic - #13
Participants could provide the dollar amount for this item.
Recommendation(s):
•

Add a ‘Don’t know’ check box to be consistent with previous questions.

Finding #15: Working from home
In testing this question performed well.
Recommendation(s):
•

No recommendations needed.

Finding #16: Percent of employees working from home
Most participants were thinking about how they would answer this question for 2020. Many were
unsure how they would answer if employees worked from home for only part of the year. Some
participants also stated that they do not have a formal way of tracking this information. One respondent
9
DRB Clearance Number: CBDRB-FY21-ESMD001-005

suggested asking for “the percentages of employees that work from home at the following
frequencies…” If question remains as is, answers are likely to be very rough estimates.
Recommendation(s):
•

Consider clarifying ‘for employees that worked from home for at least part of the year,’ if you
want to capture all employees. Otherwise, consider adding a threshold such as ‘for employees
who worked from home for at least # months.’

Finding #17: Factors affecting work from home
Participants were all able to answer this question. However, some participants felt that some factors
were missing from the list:
•
•
•

Access to high-speed internet and personal computer or some equipment
Office culture
No desire to work from home

Recommendation(s):
•

Consider adding suggested categories, particularly ‘Lack of high-speed internet access’ and
‘office culture.’

Finding #18: New credit applications
Most participants were able to answer this question, however, only a couple answered ‘yes.’ One
participant mentioned doing a RFP, for which there was no application.
Recommendation(s):
•

If RFP should be captured here it should be added to the examples. Otherwise, no
recommendations needed.

Finding #19: Credit sources
This question was not applicable to most of our participants. However, those that did review this
question would be able to answer it. One participant felt that ‘individuals’ was a category missing from
this list.
Recommendation(s):
•

None needed, review write-ins to see if there are any reoccurring write-ins that should be added
to categories.

Finding #20: New credit applications amount
Participants were able to answer this question.
10
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Recommendation(s):
•

Change ‘sought’ to ‘requested’ to be consistent with previous questions.

Finding #21: Business financing uses
Participants were able to answer this question. One participant mentioned that not all businesses know
what capital assets are and suggested we add “equipment” in parenthesis.
Recommendation(s):
•

Consider adding examples such as ‘equipment and structures’ to the capital assets category.

B. Management Practices
Finding #22: Managing service or production problems
Most participants were able to answer this question. A few participants said that the examples here
greatly relate to a production problem, but for them it would be a service issue, a missed process or
procedure.
Recommendation(s):
•

Consider revising to ‘production problem or service issue.’

Finding #23: Number of key performance indicators
In testing this question performed well.
Recommendation(s):
•

No recommendations needed.

Finding #24: Frequency of key performance indicators - #3
In testing this question performed well.
Recommendation(s):
•

No recommendations needed.

Finding #25: Frequency of key performance indicators - #4
In testing this question performed well.
Recommendation(s):
11
DRB Clearance Number: CBDRB-FY21-ESMD001-005

•

No recommendations needed.

Finding #26: Business targets - #5
In testing this question performed well.
Recommendation(s):
•

No recommendations needed.

Finding #27: Business targets - #6
Most participants were able to answer this question. However, most questioned to what they would be
comparing their answers to.
Recommendation(s):
•

Consider revising to: ‘In 2019, how much effort did it take to achieve business, service, or
production targets at this business as compared to previous years’ effort?’

Finding #28: Employee promotion
In testing this question performed well.
Recommendation(s):
•

No recommendations needed.

Finding #29: Under-performing employee
Some participants struggled to answer this question. These participants took issue with the 6 month
timeframe. Many stated that it can take much longer to document an under-performing employee
before taking action. Other participants stated that the timeframe could depend on the type of
employee.
Recommendation(s):
•

Consider removing the word ‘when’ from the question and combine the first two answer
categories into a simple ‘yes’ answer.

Finding #30: Employee training expenditures
Some participants felt that they would be able to answer this question, however, several participants
were unsure if this was a number that was readily available in their records. Most respondents would
also not include the salary of full-time trainers when answering this question.
Recommendation(s):
12
DRB Clearance Number: CBDRB-FY21-ESMD001-005

•
•

The question is fine as written but will likely result in very rough estimations or guesses.
Consider adding an instruction to include salaries of full-time trainers, if appropriate.

Finding #31: Employee training participation - #10
Most participants felt that they would be able to answer this question with some research. Most
participants interpreted ‘workers’ to mean W-2 employees.
Recommendation(s):
•
•

Consider changing ‘worker’ to ‘employee,’ which would also be consistent with rest of form.
Add ‘contractors’ if you want respondents to include them.

Finding #32: Employee training participation - #11
Most participants were unsure if this number was readily available in their records. Most participants
believed that they could only answer with a rough estimate. One participant felt that they should have
been able to skip this question if they answered ‘0’ to the previous question.
Recommendation(s):
•
•

Answers to this question would likely be a rough estimates.
Consider adding a skip if respondents answer ‘0’ to previous question.

Finding #33: Employee training participation - #12
Participants were all able to answer this question. However, some participants felt that some factors
were missing from the list:
•
•
•
•
•
•
•
•
•
•
•

Trainings required for fed state and local compliance (i.e., sexual assault harassment training
prevention),
OSHA required training for insurance purposes (e.g. to operate forklift or golf cart)
Office culture.
Management development training
Financial training
Leadership skills
Crucial conversations training
Conflict training
Training on updates to rules and regulations
Training on new software
Safety Training

Recommendation(s):
•
•

Consider adding suggested categories.
Consider adding an ‘other – specify’ category.

13
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Appendix A: Draft Questions, Round 1
A. FINANCING
CAPITAL FUNDING

1. What were the source(s) of capital used to start or initially acquire this business? Select
all that apply.
Personal/family savings of owner(s)
Personal/family assets other than savings of owner(s)
Personal/family home equity loan
Personal credit card(s) carrying balances
Business credit card(s) carrying balances
Government-guaranteed business loan from a bank or financial institutions, including
SBA-guaranteed loans
Business loan from a bank or financial institution
Business loan from a Federal, state, or local government
Business loan/investment from family/friend(s)
Investment by venture capitalist(s)
Grants
Other source(s) of capital
Don’t know
None needed (Go to Q3)

AMOUNT OF CAPITAL NEEDED TO START OR INITIALLY ACQUIRE THE BUSINESS

2. What was the total amount of capital used to start or initially acquire this business?
Capital includes savings, other assets, and borrowed funds of owner(s).
o
o
o
o
o
o
o
o

$0
$1 - $4,999
$5,000 - $9,999
$10,000 - $24,999
$25,000 - $49,999
$50,000 - $99,999
$100,000 - $249,999
$250,000 or more
14
DRB Clearance Number: CBDRB-FY21-ESMD001-005

o Don’t know

15
DRB Clearance Number: CBDRB-FY21-ESMD001-005

FUNDING FROM OWNER(S)

3. In 2019, what was the total amount of money that the owner(s) personally put into this
business? Your best estimate is fine.
Include:

•
•
•
•
•
o
o
o
o
o
o
o
o
o
o

Investments from personal savings
Personal retirement accounts
Home equity loans
Personally borrowed funds
Personal credit cards
$0
$1 - $4,999
$5,000 - $9,999
$10,000 - $24,999
$25,000 - $49,999
$50,000 - $99,999
$100,000 - $249,999
$250,000 or more
Don’t know
Business does not have owners

PROFITABILITY

4. For the last 12 months ending on December 31, 2019, did this business have profits,
losses, or break even?
o Profits
o Losses
o Break even

16
DRB Clearance Number: CBDRB-FY21-ESMD001-005

NEGATIVE IMPACT ON PROFITABILITY

5. For the last 12 months ending on December 31, 2019, which of the following negatively
impacted the profitability of this business? Select all that apply.
Lack of access to financial capital
High cost of financial capital
Unable to find qualified labor
Taxes
Government regulations (for example, U.S. Federal, state and/or local)
Slow business or lost sales
Customers or clients not making payments or paying late
The unpredictability of business conditions
Changes or updates in technology
None of the above

OPERATIONAL CHANGES AS A RESULT OF CORONAVIRUS PANDEMIC

When answering questions in this section, please factor in the 2020 coronavirus pandemic and how it
affected your business.

6. How would you assess the overall affect of the 2020 coronavirus pandemic on this
business’s sales in 2020?
o
o
o
o
o

Increased significantly
Increased somewhat
Had little to no change
Decreased somewhat
Decreased significantly
17
DRB Clearance Number: CBDRB-FY21-ESMD001-005

7. Since March 13, 2020, has this business requested financial assistance from any of the following
sources?

Include:
• Paycheck Protection Program (PPP) loans
• Small Business Economic Injury Disaster Loans
• Other Federal, state, or local government programs aimed specifically at
addressing the coronavirus pandemic
o Yes
o No (Go to Q11)
8. What was the total dollar amount of government-sponsored coronavirus pandemic
financial assistance (from the previous question 7) that this business sought?
$________________.00

9. What was the dollar amount this business received? If no financial assistance was
received, enter ‘0’ and go to Question 11.
$_______________.00

10. What share of this government-sponsored coronavirus pandemic financial assistance, if
any, was forgiven or does not need to be paid back?
o
o
o
o
o

None
1-25 %
26-50 %
51-75 %
76-100 %

11. Will this business be able to meet its funding needs to maintain operations into the
next year (2021)?
o Yes
o No
o Don't know

12. In 2020, how concerned are you about the financial health of this business?
o Not concerned at all
18
DRB Clearance Number: CBDRB-FY21-ESMD001-005

o Somewhat concerned
o Very concerned

13. What is the total amount of outstanding debt of this business?
o
o
o
o
o
o
o
o

No debt
$1-$25,000
$25,001 - $50,000
$50,001 - $100,000
$100,001 - $250,000
$250,001 - $1,000,000
More than $1,000,000
Don’t know

WORKING FROM HOME

14. In 2019, did this business allow any employees to work from home?
o Yes
o No – (Go to Q16)
PERCENT OF EMPLOYEES WORKING FROM HOME

15. In 2019, what percent of all employees at this business worked from home at the
following frequencies? If none report 0. Estimates are acceptable.
•
•
•
•
•

Never
Occasionally (less than one day per week)
One day per week
Two to four days per week
Five days per week (full-time)

________%
________%
________%
________%
________%
TOTAL= 100%

FACTORS AFFECTING WORKING FROM HOME

16. In 2019, what factors limited the ability of this business’s employees to work from
home? Select all that apply.
o Job or parts of job cannot be performed from home
o Management of employees working from home too costly or complicated
19
DRB Clearance Number: CBDRB-FY21-ESMD001-005

o Security (IT or other) concerns
o Other
o No factors

SMALL BUSINESS CREDIT

In this section, answer based on this business’s credit seeking activity in the last 12 months.
Please exclude loans that were forgiven as part of a government response to the coronavirus
pandemic.
NEW CREDIT APPLICATIONS

17. For the 12 months ending on December 31, 2019, did this business submit an
application for new credit (for example, a credit card, loan, line of credit, trade
financing, etc.)?
o Yes
o No (Go to Section C)
o Don’t know (Go to Section C)
CREDIT SOURCES

18. For the credit this business applied for in the 12 months ending on December 31, 2019,
from what lender(s) did this business apply for new credit? Select all that apply.
Bank (financial institution that accepts deposits and extends credit, either in person or
online)
Credit Union (non-profit financial institution that provides banking services, including
loans, to its members)
Fintech/Online Lender (type of finance company that operates exclusively online
and/or by phone to issue loans or lines of credit, usually with no collateral
requirements. Includes payments companies that offer loans and cash advances to
merchants)
Finance Company (company that provides loans or leases but unlike a bank does not
accept deposits or offer banking services. Examples include mortgage companies,
equipment dealers, and auto finance companies)

20
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Community Development Financial Institution (CDFI) (financial institution certified by
the U.S. Department of the Treasury that provides credit, financial services, and
technical assistance to underserved people or places)
Other, specify:___________________________
NEW CREDIT APPLICATIONS AMOUNT

19. For the 12 months ending on December 31, 2019, how much of the total amount of
credit sought did this business obtain?
o None
o Some
o All
o Don’t know

BUSINESS FINANCING USES

20. For the 12 months ending on December 31 st, 2019, for what purpose(s) was this
business seeking financing? Select all that apply.
Meet operating expenses
Expand business, pursue new opportunity, or acquire business assets
Replace capital assets or make repairs
Refinance or pay down debt
Other, specify: ________________________________

B. MANAGEMENT PRACTICES MODULE
MANAGING SERVICE OR PRODUCTION PROBLEMS

1. In 2019, what best describes what happened at this business when a service or
production problem arose? For example, finding a quality defect in a product or piece of
equipment breaking down.
o We fixed it but did not take further action
o We fixed it and took action to make sure that it did not happen again
o We fixed it and took action to make sure that it did not happen again, and had a
continuous improvement process to anticipate problems like these in advance
o No action was taken
o No service or production problem arose

NUMBER OF KEY PERFORMANCE INDICATORS

21
DRB Clearance Number: CBDRB-FY21-ESMD001-005

2. In 2019, how many key performance indicators were monitored at this business? Key
performance indicators are formal, quantifiable measures of performance or quality at
this business.
o 1-2 key performance indicators
o 3-9 key performance indicators
o 10 or more key performance indicators
o No key performance indicators

FREQUENCY OF KEY PERFORMANCE INDICATORS

3. During 2019, how frequently were the key performance indicators reviewed at this
business? Select all that apply.
 Yearly
 Quarterly
 Monthly
 Weekly
 Daily
 Hourly or more frequently
 Never

4. How often does this business monitor customer satisfaction through analysis of
complaints, customer satisfaction surveys, focus groups, or other methods? Select all
that apply.
 Yearly
 Quarterly
 Monthly
 Weekly
 Daily
 Hourly or more frequently
 Never

BUSINESS TARGETS

5. In 2019, what best describes the timeframe of business, service, or production targets at
this business?
Select ONE only. Examples of business, service, or production targets include number of
customers, production, quality, efficiency, sales, waste, and on-time delivery.
22
DRB Clearance Number: CBDRB-FY21-ESMD001-005

o
o
o
o

Main focus was on short term (less than one year) targets
Main focus was on long term (more than one year) targets
Combination of short-term and long-term targets
No targets – Skip to Employee Promotion

6. In 2019, how much effort did it take to achieve business, service, or production targets
at this business? Select ONE only.
o Minimal effort
o Less than normal effort
o Normal effort
o More than normal effort
o Extraordinary effort

EMPLOYEE PROMOTION

7. In 2019, what was the primary way employees were promoted at this business?
o Promotions were based solely on performance and ability
o Promotions were based partly on performance and ability and partly on other
factors (for example, tenure or family connections)
o Promotions were based mainly on factors other than performance and ability (for
example, tenure or family connections)
o Employees are not normally promoted.

UNDER-PERFORMING EMPLOYEE

8. In 2019, when was an under-performing employee reassigned or dismissed?
o Within 6 months of identifying employee under-performance
o After 6 months of identifying employee under-performance
o Under-performing employees are not reassigned or dismissed
o No under-performing employees identified

EMPLOYEE TRAINING EXPENDITURES

9. In 2019, what did your business spend on formal training programs?
$

.00

23
DRB Clearance Number: CBDRB-FY21-ESMD001-005

EMPLOYEE TRAINING PARTICIPATION

10. In 2019, what percent of or how many workers received any formal training?
__________%

OR

Number of workers: ___________

11. In 2019, what was the average number of formal training hours each worker received?
Average number of hours: ___________

12. Which of the following areas did workers cover during their training? Select all that
apply.






Improving teamwork or problem-solving skills
Training in sales and customer services
Training to use computers and/or new equipment
Training on the same use of equipment and tools
None of the above

24
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Appendix B: Interview Protocol, Round 1
Cognitive Interviewing Guide: Draft Content for 2021 ABS (4 th Year)
Interviewer notes are in red.
Introduction:
• Explain purpose of meeting: to obtain feedback on reporting to the ABS, to understand the
process of answering questions, and to evaluate how new questions work or don’t work.
• Some questions will ask may seem odd and/or obvious, but we don’t want to assume we know
what you are thinking.
• Explain that we are not testing the respondent – we only want to evaluate the questionnaire
• Structure of meeting: Understand your business and your role in it, then review the
questionnaire and ask you questions as you complete it.
• Ask respondent to sign the consent form.
Research Questions to Address:
• Are respondents able to answer the questions as intended?
• Are there any items that cause the respondents problems?
o What can be done to fix those problems?
• Are there any order/context effects?
Before the Questionnaire:
• What types of goods or services does this business provide?
• What is your role in the company? What kind of responsibilities do you have?
• Are you one of the owners of the business?
• Have you completed other surveys from the Census Bureau or other agencies? If so, which
ones?
• What was your role in the process for responding to this questionnaire? (Gather data? Enter
data? Consult with data providers? Etc.)
Thanks for providing some background information. Now let’s look at the survey questions.
Direct participant to look at draft content.

FINANCING MODULE - ABS (4th Year)
Question 1
• How would you answer this question?
25
DRB Clearance Number: CBDRB-FY21-ESMD001-005

•
•
•
•
•

Are there any categories we are missing?
Did the current owner(s) start this business?
How long ago was this business started?
Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
Is this something you would have in your records?

Question 2
• How would you answer this question?
• Are you able to fit your answer into the provided categories?
• If this question asked for an estimate of the dollar amount of capital needed to start this
business would you be able to provide that figure?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 3
• How would you answer this question?
• Are you able to fit your answer into the provided categories?
• If this question asked for the dollar amount would you be able to provide that figure?
• Are there other sources of money you would include in your answer?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 4
• How would you answer this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 5
• How would you answer this question? Note how R answered previous question.
• Are the categories clear or unclear?
• Can you think of anything else that impacted the profitability of this company that is not listed
here?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 6
• How would you answer this question?
• What types of impacts were you considering when answering this question?
26
DRB Clearance Number: CBDRB-FY21-ESMD001-005

•
•
•

In your opinion, what is the difference between ‘significant’ and ‘somewhat?’
Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
Is this something you would have in your records?

Question 7
• How would you answer this question?
• Is the list of includes helpful or unhelpful?
• Can you think of any other loans or grants you would include when answering? (Note if R
includes other kinds of government programs that may not be specifically COVID-19 related)
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 8
• How would you answer this question?
• How easy or difficult is it to provide this figure?
• Would you prefer to provide an open-ended dollar amount field or a range for this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 9
• How would you answer this question?
• How easy or difficult is it to provide this figure?
• If R has not received any money: Why?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 10
• How would you answer this question?
• How easy or difficult is it to answer this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 11
• How would you answer this question?
• How easy or difficult is it to answer this question?
• What factors were you considering when answering this question?
• Would you need to reach out to other people or departments to answer this question?
27
DRB Clearance Number: CBDRB-FY21-ESMD001-005

•

o IF YES: What departments/positions would you reach out to?
Is this something you would have in your records?

Question 12
• How would you answer this question?
• How easy or difficult is it to answer this question?
• What factors were you considering when answering this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 13
• How would you answer this question?
• If this question asked for the dollar amount would you be able to provide that figure?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 14
• How would you answer this question?
• What does “work from home” mean to you?
• Could you tell me a little bit about your company’s work from home programs or policies?

Question 15
• How would you answer this question?
• How did you come up with your answer?
• Would you need to reach out to anyone else in your company to answer this question? If YES,
what is the role/position of the person(s) you’d need to reach out to?
• Is this something that you would have in your records?
Question 16
• How would you answer this question?
• In your own words, what is this question asking?
• What factors were you considering when answering this question?
• Were you thinking only of negative effects? Or were you also thinking of any positive effects?
• Is this something that you have in your records?
• If R selected Other, what other factors were you thinking about?

28
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Question 17
• How would you answer this question?
• How would you define ‘new credit?’
• Are the examples helpful or unhelpful?
• Up until now, we have been asking you about this company’s activities in 2019, but this
question asks about the ‘last 12 months.’ Did you take that into account when answering this
question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 18
• How would you answer this question?
• Are the categories clear or unclear?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 19
• How would you answer this question?
• If this question asked for the dollar amount would you be able to provide that figure?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 20
• How would you answer this question?
o IF OTHER: What would put in other?
• Are the categories clear or unclear?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

MANAGEMENT PRACTICES MODULE - ABS (4th Year)
Skip management practices section if running low on time.
Question 1
• How would you answer this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
29
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Question 2
• How would you answer this question? Note if R has issues/concerns with “key performance
indicators”.
o IF YES: Can you give me any examples?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
• Are you familiar with the term “key performance indicator?” Is this a term that your business
uses?
Question 3
• How would you answer this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 4
• How would you answer this question?
o IF YES: Can you give me any examples of metrics and/or websites that you monitor?
▪ Is this monitoring a formal process?
o IF NO: Does your company monitor customer satisfaction in any informal way?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 5
• How would you answer this question?
o IF YES: Can you give me examples of business targets?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 6
• How would you answer this question?
• How would you define ‘effort?’
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 7
• How would you answer this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 8
30
DRB Clearance Number: CBDRB-FY21-ESMD001-005

•
•
•
•
•
•

How would you answer this question?
How would your company define ‘underperforming?’
How would you define ‘reassigned?’
Does your company have procedures in place for dealing with an underperformer?
o IF YES: Can you give me examples of the types of actions you might take?
Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
Is this something you would have in your records?

Question 9
• How would you answer this question?
• How does your company define ‘formal training programs?’
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 10
• How would you answer this question?
• Do you think it would be easier or more difficult to answer this question with a count as
opposed to a percentage?
• When answering this questions were you thinking only of W-2 employees, or other types of
employees as well?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 11
• How would you answer this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
Question 12
• How would you answer this question?
• Can you think of other types of training that we do not have listed?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
•
•

Having gone through this set of management questions, do you feel that you are still the most
appropriate person to answer these questions?
o If not, who do you feel would be responsible for this section?
How long do you think it would take you to complete these questions?

31
DRB Clearance Number: CBDRB-FY21-ESMD001-005

WRAP UP
• Do you have any other comments of suggestions about anything we have discussed today?
THANK YOU FOR YOUR TIME!

32
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Appendix C: Draft Questions, Round 2
A. FINANCING
CAPITAL FUNDING
21. What were the source(s) of capital used to start or initially acquire this business? Select
all that apply.
Personal/family savings of owner(s)
Personal/family assets other than savings of owner(s)
Personal/family home equity loan
Personal credit card(s) carrying balances
Business credit card(s) carrying balances
Government-guaranteed business loan from a bank or financial institutions, including
SBA-guaranteed loans
Business loan from a bank or financial institution
Business loan from a Federal, state, or local government
Business loan/investment from family/friend(s)
Investment by venture capitalist(s)
Grants
Other source(s) of capital
Don’t know
None needed (Go to Q3)
AMOUNT OF CAPITAL NEEDED TO START OR INITIALLY ACQUIRE THE BUSINESS
22. What was the total amount of capital used to start or initially acquire this business?
Capital includes savings, other assets, and borrowed funds of owner(s).
o
o
o
o
o
o
o
o
o

$0
$1 - $4,999
$5,000 - $9,999
$10,000 - $24,999
$25,000 - $49,999
$50,000 - $99,999
$100,000 - $249,999
$250,000 or more
Don’t know

33
DRB Clearance Number: CBDRB-FY21-ESMD001-005

FUNDING FROM OWNER(S)
23. In 2019, what was the total amount of money that the owner(s) personally put into this
business? Your best estimate is fine.
Include:
• Investments from personal savings
• Personal retirement accounts
• Home equity loans
• Personally borrowed funds
• Personal credit cards
o
o
o
o
o
o
o
o
o
o
o

$0
$1 - $4,999
$5,000 - $9,999
$10,000 - $24,999
$25,000 - $49,999
$50,000 - $99,999
$100,000 - $249,999
$250,000 or more
Don’t know
This business is publicly traded
Business does not have owners

PROFITABILITY
24. For the last 12 months ending on December 31, 2019, did this business have profits,
losses, or break even?
o Profits
o Losses
o Break even

34
DRB Clearance Number: CBDRB-FY21-ESMD001-005

NEGATIVE IMPACT ON PROFITABILITY
25. For the last 12 months ending on December 31, 2019, which of the following negatively
impacted the profitability of this business? Select all that apply.
Lack of access to financial capital
High cost of financial capital
Unable to find or recruit qualified labor
Cost of labor
Taxes
Government regulations (for example, U.S. Federal, state and/or
local)
Slow business or lost sales
Customers or clients not making payments or paying late
The unpredictability of business conditions
Changes or updates in technology
None of the above
OPERATIONAL CHANGES AS A RESULT OF CORONAVIRUS PANDEMIC
When answering questions in this section, please factor in the 2020 coronavirus pandemic and
how it affected your business.
26. How would you assess the overall affect of the 2020 coronavirus pandemic on this
business’s sales in 2020?
o
o
o
o
o

Increased significantly
Increased somewhat
Had little to no change
Decreased somewhat
Decreased significantly

27. Since March 13, 2020, has this business requested financial assistance from any of the following
sources?

Include:
• Paycheck Protection Program (PPP) loans
• Small Business Economic Injury Disaster Loans
• Other Federal, state, or local government programs aimed specifically at
addressing the coronavirus pandemic
o Yes
35
DRB Clearance Number: CBDRB-FY21-ESMD001-005

o No (Go to Q11)
28. What was the total dollar amount of government-sponsored coronavirus pandemic
financial assistance (from the previous question 7) that this business sought? Round to
the nearest thousand dollars, if less than $500 enter $0.
$___________,000.00
29. What was the dollar amount this business received? If no financial assistance was
received, enter ‘0’ and go to Question 11. Round to the nearest thousand dollars, if less
than $500 enter $0.
$___________,000.00
30. What share of this government-sponsored coronavirus pandemic financial assistance, if
any, was forgiven or does not need to be paid back?
o
o
o
o
o

None
1-25 %
26-50 %
51-75 %
76-100 %

31. Will this business be able to maintain operations into the next year (2021)?
o
o
o
o

Yes, at or above the current level of operations
Yes, at below current levels of operations
No
Don't know

32. In 2020, how concerned are you about the financial health of this business?
o Not that concerned
o Somewhat concerned
o Very concerned
33. What is the total amount of outstanding debt of this business? Round to the nearest
thousand dollars, if less than $500 enter $0.
$___________,000.00

WORKING FROM HOME
34. In 2019, did this business allow any employees to work from home?
o Yes
o No – (Go to Q16)
36
DRB Clearance Number: CBDRB-FY21-ESMD001-005

PERCENT OF EMPLOYEES WORKING FROM HOME
35. In 2019, what percent of all employees at this business worked from home at the
following frequencies? If none report 0. Estimates are acceptable.
•
•
•
•
•

Never
Less than one day per week
One day per week
Two to four days per week
Five days per week (full-time)

________%
________%
________%
________%
________%
TOTAL= 100%

FACTORS AFFECTING WORKING FROM HOME
36. In 2019, what factors limited the ability of this business’s employees to work from
home? Select all that apply.
o
o
o
o
o

Job or parts of job cannot be performed from home
Management of employees working from home too costly or complicated
Security (IT or other) concerns
Other
No factors

SMALL BUSINESS CREDIT
In this section, answer based on this business’s credit seeking activity in the last 12 months.
Please exclude loans that were forgiven as part of a government response to the coronavirus
pandemic.
NEW CREDIT APPLICATIONS
37. For the 12 months ending on December 31, 2019, did this business submit an
application for new credit (for example, a credit card, loan, line of credit, trade
financing, etc.)?
o Yes
o No (Go to Section C)
o Don’t know (Go to Section C)

37
DRB Clearance Number: CBDRB-FY21-ESMD001-005

CREDIT SOURCES
38. For the credit this business applied for in the 12 months ending on December 31, 2019,
from what lender(s) did this business apply for new credit? Select all that apply.
Bank (financial institution that accepts deposits and extends credit, either in person or
online)
Credit Union (non-profit financial institution that provides banking services, including
loans, to its members)
Fintech/Online Lender (type of finance company that operates exclusively online
and/or by phone to issue loans or lines of credit, usually with no collateral
requirements. Includes payments companies that offer loans and cash advances to
merchants)
Finance Company (company that provides loans or leases but unlike a bank does not
accept deposits or offer banking services. Examples include mortgage companies,
equipment dealers, and auto finance companies)
Community Development Financial Institution (CDFI) (financial institution certified by
the U.S. Department of the Treasury that provides credit, financial services, and
technical assistance to underserved people or places)
Other, specify:___________________________
NEW CREDIT APPLICATIONS AMOUNT
39. For the 12 months ending on December 31, 2019, how much of the total amount of
credit sought did this business obtain?
o None
o Some
o All
o Don’t know
BUSINESS FINANCING USES
40. For the 12 months ending on December 31 st, 2019, for what purpose(s) was this
business seeking financing? Select all that apply.
Meet operating expenses
Expand business, pursue new opportunity, or acquire business assets
Replace capital assets or make repairs
Refinance or pay down debt
Other, specify: ________________________________
38
DRB Clearance Number: CBDRB-FY21-ESMD001-005

B. MANAGEMENT PRACTICES MODULE
MANAGING SERVICE OR PRODUCTION PROBLEMS
13. In 2019, what best describes what happened at this business when a service or
production problem arose? For example, finding a quality defect in a product or piece of
equipment breaking down.
o We fixed it but did not take further action
o We fixed it and took action to make sure that it did not happen again
o We fixed it and took action to make sure that it did not happen again, and had a
continuous improvement process to anticipate problems like these in advance
o No action was taken
o No service or production problem arose
NUMBER OF KEY PERFORMANCE INDICATORS
14. In 2019, how many key performance indicators were monitored at this business? Key
performance indicators are formal, quantifiable measures of performance or quality at
this business.
o 1-2 key performance indicators
o 3-9 key performance indicators
o 10 or more key performance indicators
o No key performance indicators
FREQUENCY OF KEY PERFORMANCE INDICATORS
15. During 2019, how frequently were the key performance indicators reviewed at this
business? Select all that apply.
 Yearly
 Quarterly
 Monthly
 Weekly
 Daily
 Hourly or more frequently
 Never

39
DRB Clearance Number: CBDRB-FY21-ESMD001-005

16. How often does this business monitor customer satisfaction through analysis of
complaints, customer satisfaction surveys, focus groups, or other methods? Select all
that apply.
 Yearly
 Quarterly
 Monthly
 Weekly
 Daily
 Hourly or more frequently
 Never
BUSINESS TARGETS
17. In 2019, what best describes the timeframe of business, service, or production targets at
this business?
Select ONE only. Examples of business, service, or production targets include number of
customers, production, quality, efficiency, sales, waste, and on-time delivery.
o Main focus was on short term (less than one year) targets
o Main focus was on long term (more than one year) targets
o Combination of short-term and long-term targets
o No targets – Skip to Employee Promotion
18. In 2019, how much effort did it take to achieve business, service, or production targets
at this business? Select ONE only.
o Minimal effort
o Less than normal effort
o Normal effort
o More than normal effort
o Extraordinary effort
EMPLOYEE PROMOTION
19. In 2019, what was the primary way employees were promoted at this business?
o Promotions were based solely on performance and ability
o Promotions were based partly on performance and ability and partly on other
factors (for example, tenure or family connections)
o Promotions were based mainly on factors other than performance and ability (for
example, tenure or family connections)
o Employees are not normally promoted.
o Not applicable

40
DRB Clearance Number: CBDRB-FY21-ESMD001-005

UNDER-PERFORMING EMPLOYEE
20. In 2019, when was an under-performing employee reassigned or dismissed?
o Within 6 months of identifying employee under-performance
o After 6 months of identifying employee under-performance
o Under-performing employees are not reassigned or dismissed
o No under-performing employees identified
o Not applicable
EMPLOYEE TRAINING EXPENDITURES
21. In 2019, what did your business spend on formal training programs? Round to the
nearest thousand dollars, if less than $500 enter $0.
$___________,000.00

EMPLOYEE TRAINING PARTICIPATION
22. In 2019, what percent of or how many workers received any formal training?
__________%

OR

Number of workers: ___________

23. In 2019, what was the average number of formal training hours each worker received?
Average number of hours: ___________

24. Which of the following areas did workers cover during their training? Select all that
apply.






Improving teamwork or problem-solving skills
Training in sales and customer services
Training to use computers and/or new equipment
Training on the same use of equipment and tools
None of the above

41
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Appendix D: Interview Protocol, Round 2
Cognitive Interviewing Guide: Draft Content for 2021 ABS (4 th Year)
Interviewer notes are in red.
Introduction:
• Explain purpose of meeting: to obtain feedback on reporting to the ABS, to understand the
process of answering questions, and to evaluate how new questions work or don’t work.
• Some questions will ask may seem odd and/or obvious, but we don’t want to assume we know
what you are thinking.
• Explain that we are not testing the respondent – we only want to evaluate the questionnaire
• Structure of meeting: Understand your business and your role in it, then review the
questionnaire and ask you questions as you complete it.
• Ask respondent to sign the consent form.
Research Questions to Address:
• Are respondents able to answer the questions as intended?
• Are there any items that cause the respondents problems?
o What can be done to fix those problems?
• Are there any order/context effects?
Before the Questionnaire:
• What types of goods or services does this business provide?
• What is your role in the company? What kind of responsibilities do you have?
• Are you one of the owners of the business?
• Have you completed other surveys from the Census Bureau or other agencies? If so, which
ones?
• What was your role in the process for responding to this questionnaire? (Gather data? Enter
data? Consult with data providers? Etc.)

Thanks for providing some background information. Now let’s look at the survey questions.
Direct participant to look at draft content.

FINANCING MODULE - ABS (4th Year)
Question 1
• How would you answer this question?
• Are there any categories we are missing?
42
DRB Clearance Number: CBDRB-FY21-ESMD001-005

•
•
•
•

Did the current owner(s) start this business?
How long ago was this business started?
Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
Is this something you would have in your records?

Question 2
• How would you answer this question?
• Are you able to fit your answer into the provided categories?
• If this question asked for an estimate of the dollar amount of capital needed to start this
business would you be able to provide that figure?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 3
• How would you answer this question?
• Are you able to fit your answer into the provided categories?
• If this question asked for the dollar amount would you be able to provide that figure?
• Are there other sources of money you would include in your answer?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 4
• How would you answer this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 5
• How would you answer this question? Note how R answered previous question.
• Are the categories clear or unclear?
• Can you think of anything else that impacted the profitability of this company that is not listed
here?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 6
43
DRB Clearance Number: CBDRB-FY21-ESMD001-005

•
•
•
•
•

How would you answer this question?
What types of impacts were you considering when answering this question?
In your opinion, what is the difference between ‘significant’ and ‘somewhat?’
Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
Is this something you would have in your records?

Question 7
• How would you answer this question?
• Is the list of includes helpful or unhelpful?
• Can you think of any other loans or grants you would include when answering? (Note if R
includes other kinds of government programs that may not be specifically COVID-19 related)
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 8
• How would you answer this question?
• How easy or difficult is it to provide this figure?
• Would you prefer to provide an open-ended dollar amount field or a range for this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 9
• How would you answer this question?
• How easy or difficult is it to provide this figure?
• If R has not received any money: Why?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 10
• How would you answer this question?
• How easy or difficult is it to answer this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 11
• How would you answer this question?
44
DRB Clearance Number: CBDRB-FY21-ESMD001-005

•
•
•
•

How easy or difficult is it to answer this question?
What factors were you considering when answering this question?
Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
Is this something you would have in your records?

Question 12
• How would you answer this question?
• How easy or difficult is it to answer this question?
• What factors were you considering when answering this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 13
• How would you answer this question?
• If this question asked for the dollar amount would you be able to provide that figure?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 14
• How would you answer this question?
• What does “work from home” mean to you?
• Could you tell me a little bit about your company’s work from home programs or policies?

Question 15
• How would you answer this question?
• How did you come up with your answer?
• Would you need to reach out to anyone else in your company to answer this question? If YES,
what is the role/position of the person(s) you’d need to reach out to?
• Is this something that you would have in your records?

Question 16
• How would you answer this question?
• In your own words, what is this question asking?
• What factors were you considering when answering this question?
• Were you thinking only of negative effects? Or were you also thinking of any positive effects?
• Is this something that you have in your records?
• If R selected Other, what other factors were you thinking about?
45
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Question 17
• How would you answer this question?
• How would you define ‘new credit?’
• Are the examples helpful or unhelpful?
• Up until now, we have been asking you about this company’s activities in 2019, but this
question asks about the ‘last 12 months.’ Did you take that into account when answering this
question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 18
• How would you answer this question?
• Are the categories clear or unclear?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 19
• How would you answer this question?
• If this question asked for the dollar amount would you be able to provide that figure?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 20
• How would you answer this question?
o IF OTHER: What would put in other?
• Are the categories clear or unclear?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

MANAGEMENT PRACTICES MODULE - ABS (4th Year)
Question 1
• How would you answer this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
46
DRB Clearance Number: CBDRB-FY21-ESMD001-005

•

Is this something you would have in your records?

Question 2
• How would you answer this question? Note if R has issues/concerns with “key performance
indicators”.
o IF YES: Can you give me any examples?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
• Are you familiar with the term “key performance indicator?” Is this a term that your business
uses?

Question 3
• How would you answer this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 4
• How would you answer this question?
o IF YES: Can you give me any examples of metrics and/or websites that you monitor?
▪ Is this monitoring a formal process?
o IF NO: Does your company monitor customer satisfaction in any informal way?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 5
• How would you answer this question?
o IF YES: Can you give me examples of business targets?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 6
• How would you answer this question?
• How would you define ‘effort?’
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

47
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Question 7
• How would you answer this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 8
• How would you answer this question?
• How would your company define ‘underperforming?’
• How would you define ‘reassigned?’
• Does your company have procedures in place for dealing with an underperformer?
o IF YES: Can you give me examples of the types of actions you might take?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 9
• How would you answer this question?
• How does your company define ‘formal training programs?’
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 10
• How would you answer this question?
• Do you think it would be easier or more difficult to answer this question with a count as
opposed to a percentage?
• When answering this questions were you thinking only of W-2 employees, or other types of
employees as well?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

Question 11
• How would you answer this question?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?
48
DRB Clearance Number: CBDRB-FY21-ESMD001-005

Question 12
• How would you answer this question?
• Can you think of other types of training that we do not have listed?
• Would you need to reach out to other people or departments to answer this question?
o IF YES: What departments/positions would you reach out to?
• Is this something you would have in your records?

•
•

Having gone through this set of management questions, do you feel that you are still the most
appropriate person to answer these questions?
o If not, who do you feel would be responsible for this section?
How long do you think it would take you to complete these questions?

WRAP UP
• Do you have any other comments of suggestions about anything we have discussed today?

THANK YOU FOR YOUR TIME!

49
DRB Clearance Number: CBDRB-FY21-ESMD001-005


File Typeapplication/pdf
AuthorKenneth Herrell (CENSUS/ESMD FED)
File Modified2020-11-30
File Created2020-11-30

© 2024 OMB.report | Privacy Policy