Non-Substantive Change Request
Small Business Pulse Survey
OMB Number 0607-1014
May 14, 2020
Per the terms of clearance included on the April 22, 2020 Notice of OMB Action, the Census Bureau is writing to request approval of a non-substantive change to the Small Business Pulse Survey (SBPS). This non-substantive change request summarizes recommendations based on cognitive testing for changes to the SBPS to be considered for Week 4 and describes the proposed changes the Census Bureau is seeking for implementation with Week 4 collection beginning on May 17th.
Following a brief discussion of our organizing principles, assumptions, and criteria for evaluation, we provide the proposed non-substantive changes to be implemented in Week 4 of the SBPS collection. We discuss these within the context of data evaluation based on our first week of collection, and then provide additional details by question.
Organizing Principle:
We divided the questions into two groups (targeted and non-targeted). Each group is addressed in Part 1 and Part 2 of this memo:
(1) Targeted: These are the three questions (1, 3, 15) that OMB specifically targeted in our discussion on April 17, 2020 and for which Census agreed to provide cognitive testing.
(2) Non-Targeted: These are all other questions (2, 4-14,16).
Part 3 of this memo includes the proposed revised instrument in its entirety.
Assumptions:
The cognitive testing report produced by Diane K. Willimack, Methodology Director for Measurement & Response Improvement, Amy E. Anderson Riemer, Chief, Data Collection Methodology & Research Branch (DCMRB) and Members of DCMRB, U.S. Census Bureau, was reviewed by Catherine Buffington, Special Projects and Research, Economic Directorate, and Lucia Foster, Chief Economist, U.S. Census Bureau, using the following assumptions based upon what is feasible based upon the work required to ensure consistency across instrument, collection mechanism, and dissemination tools:
Changes to the text of questions are in-scope for consideration for Week 4
The following are out-of-scope for Week 4:
Changes in the number of questions (adds/deletes)
Changes in the ordering of questions
Changes in the number of response categories (adds/deletes)
Several factors were considered when selecting these criteria. Changes to question and response text have minimal impact on the collection instrument, processing system, and visualizations as these are text labels for which changes are straightforward. While changes to question and response text must be reviewed throughout the collection, processing, and dissemination pipeline, they do not represent changes to software or the underlying database model and thus represent minimal risk to the collection.
Proposed changes for week 4 are limited in scope to wording changes. Other modifications, including the addition or deletion of questions or response categories were not considered for implementation at this time due to schedule and resource constraints. High-level estimates indicate 6 weeks are needed to develop, test and implement content changes of this type.
Evaluation Criteria:
Each recommendation from the cognitive testing staff based on the results of the cognitive testing was evaluated by the Catherine Buffington and Lucia Foster with guidance from Kristin Stettler from the DCMRB using the following criteria:
Will the recommendation change the concept we are seeking to measure?
If yes, does this change in concept outweigh the loss of data / break in time series?
Will the rewording have other unintended consequences?
Will the work associated with the change for week 4 be outweighed by the benefit of an improvement in data quality?
Evaluating Week1 responses:
Preliminary evidence, including overall response rate, item response rate, survey completion rate, applicability across sectors, and logical consistency, from Week 1 collection of the SBPS suggests that the collection is successful. We reviewed the cognitive testing results and recommendations with this in mind.
The SBPS overall
response rate for Week 1 was 22 %, which exceeded our target response
rate of 20%. We believe this supports our initial thoughts that small
businesses would be willing to respond to the SBPS under the current
circumstances and that they find the SBPS an important way to
communicate their hardship and needs to the government. Excluding
the optional “Remarks” section of the instrument and the
EIN collection, item non-response rates are well below 1%; fewer than
1/10th of a percent do not complete all of these
items.
Leading in to cognitive testing, concerns were raised about the applicability of certain operations questions (supply chain disruptions, change in goods or services produced, and changing mode of delivery to only carryout/curbside pickup/delivery). Preliminary review of responses to these by sector seem to support that generally these questions are “working” in the sense that there is between sector variation and low impact on responses in sectors that we did not anticipate widespread impact or responses relative to sectors that we did anticipate impact/response.
Questions on program requests and receipts were evaluated for logical consistency. For each of the responses that are represented in question 14 (PPP, EIDL, SBA loan forgiveness, and Other federal) responses were checked to ensure that no one reported the receiving assistance that was not applied for. No cases failed these edits.
Original text:
Overall, how has this business been affected by the COVID-19 pandemic?
Proposed text:
Overall, how has this business been affected financially by the COVID-19 pandemic?
Decision:
Do not change question 1.
Details:
The recommendation from the cognitive testing staff was to add the word “financially” to the question in order to frame the question for respondents to one consistent concept, finances.
However, this question was written to capture the overall experience of the respondent, both with respect to finance but also to other concepts (employment, supply chain, adaptability). The testing staff find that people incorporate different aspects of the impact on their business in formulating their answer and view this as a problem with the question, and this varies across respondents. However, this subjective well-being question was written for this purpose. If Census finds a broad pattern of disagreement between the holistic sentiment provided in Q1 with the detailed questions concerning operations, that could suggest that the areas that Census decided to focus upon are missing a critical component in the well-being of businesses.
Original text:
In the last week, what was the total operating revenue for this business?
Proposed text:
In the last month, what were the total operating revenues/sales/receipts for this business, not including any financial assistance or loans?
Decision:
Accept changes.
Details:
As this question is for internal program use only, the issue of breaking the time series for public data products is not relevant.
The cognitive testing report included the following recommendations for this question:
Consider using one month as the reference period for this question or providing respondents with a choice to report revenues on a weekly or monthly basis.
Clarify or use alternative language to replace the term “operating revenue,” such as “operating revenue/sales/receipts.”
Incorporate an instruction about including or excluding government assistance.
These recommendations were motivated by the following:
Respondents indicated that they would not refer to records in order to answer this question; however, using a reference period of one week is difficult and unnatural for many respondents, and resulted in some creative response strategies for estimating revenue.
The term “Operating Revenue” was unfamiliar for some businesses. Some respondents were confused whether “operating revenue” was equivalent to gross revenue, or whether it referred to funds currently in the bank, or weren’t sure whether this was the same as sales.
In addition, financial assistance from the government affected respondents’ answers. Although most knew that this assistance was intended to support payroll, some thought it should not be factored into their revenue amount, while others did include those funds in their answers.
Original text:
In your opinion, how much time do you think will pass before this business returns to its usual level of operations?
Select only one:
☐ 1 month or less
☐ 2-3 months
☐ 4-6 months
☐ More than 6 months
☐ I do not believe this business will return to its usual level of operations
☐There has been little or no effect on this business’s usual level of operations
Proposed text:
In your opinion, how much time do you think will pass before this business returns to its normal level of operations relative to one year ago?
Select only one:
☐ 1 month or less
☐ 2-3 months
☐ 4-6 months
☐ More than 6 months
☐ I do not believe this business will return to its normal level of operations relative to one year ago
☐There has been little or no effect on this business’s normal level of operations relative to one year ago
Decision:
Change the text of question 15 based on the cognitive testing findings
Details:
The recommendation to add a reference period of one year ago for the question addresses possible seasonality issues. The change from a specific reference date (March 1, 2020) to some concept of usual or normal prior to the mailout received considerable attention. Ultimately, the term usual was used for the week 1 collection instead of the reference date to attempt addressing the seasonality concerns that were flagged. However, cognitive testing found respondents favored the term normal compared to usual. Thus, the decision is to use a reference period of one year ago and the term normal with week 4 collection.
The cognitive testing report included the following as motivation for the proposed changes:
Respondents described a variety of factors that they considered when answering this question. These included revenue, work operations, dependencies on other industries, and methods for communicating with clients and co-workers. Some pointed to different requirements and limitations for re-opening that would not be considered in “usual” operations, such as reducing capacity, wearing masks, and increases in virtual meetings and communication. Thus, many used the word “normal” when discussing this question, referring to a “new normal.” Some businesses missed their “seasons” because of COVID and expressed inability to judge being able to return to “usual” until their “season” next year. Many said their answer would depend on their state’s plans for lifting “stay at home” orders and re-opening their economy. Respondents across industries commonly referred back to the same period in 2019 as a point of reference for this question.
Original text:
In the last week, did this business experience a change in operating revenues?
Proposed text:
In the last week, did this business experience a change in operating revenues/sales/receipts, not including any financial assistance or loans?
Details:
The proposal to change language in question 2 is to maintain consistency with the proposed change to question 3 based on cognitive testing results.
Original text:
How would you describe the current availability of cash on hand for this business? Currently, cash on hand will cover:
Proposed text:
How would you describe the current availability of cash on hand for this business, including any financial assistance or loans? Currently, cash on hand will cover:
Details:
Clarification is needed whether cash on hand should include assistance related to the pandemic.
As the concept was designed to measure cash on hand which impacts the ability of a business to continue operations, financial assistance or loans received should be included.
The cognitive testing report found that respondents were not sure how to handle loans or assistance in the context of this question. Several respondents across industries were uncertain whether Federal/state/local government aid, loans, or other types of assistance should be included in “cash on hand” when formulating their response. Respondents also wondered if this question assumed that current shelter in place conditions remained in place. The report recommends adding clarification about how respondents should handle assistance or loans received, preferably in the body of the question, or, alternatively, in a clarifying note.
As cash on hand refers to all of the cash available to a business, the modification includes instructions to include financial assistance or loans received.
Original text:
The White House declared a national emergency because of the COVID-19 pandemic on March 13, 2020. Since then, has this business missed any loan payments?
Proposed text:
The White House declared a national emergency because of the COVID-19 pandemic on March 13, 2020. Since then, has this business missed any loan payments? Loan payments that have been forgiven or postponed should not be considered to be missed.
Details:
The cognitive testing staff reported that respondents were unclear how to evaluate this question with respect to payment forgiveness or postponement and recommended that language be added about reprieves or alternative payment plans.
Original text:
Since March 13, 2020, has this business missed any other scheduled payments, not including loans? Examples of other scheduled payments include payroll, rent, and utilities.
Proposed text:
Since March 13, 2020, has this business missed any other scheduled payments, not including loans? Examples of other scheduled payments include rent, utilities, and payroll. Scheduled payments that have been forgiven or postponed should not be considered to be missed.
Details:
The cognitive testing staff reported that respondents were unclear how to evaluate this question with respect to payment forgiveness or postponement and recommended that language be added about reprieves or alternative payment plans.
Proposed changes to response text:
Question 13. Since March 13, 2020, has this business requested financial assistance from any of the following sources?
Select all that apply:
☐ Paycheck Protection Program (PPP)
☐ Economic Injury Disaster Loans (EIDL)
☐ Small Business Administration (SBA) Loan Forgiveness
☐ Other Federal programs
☐ State or local government programs
☐ Banks
☐ Self
☐ Family or friends
☐ Other sources
☐ This business has not requested financial assistance from any source since March 13, 2020
Question 14. Since March 13, 2020, has this business received financial assistance from any of these programs from the Federal government?
Select all that apply:
☐ Paycheck Protection Program (PPP)
☐ Economic Injury Disaster Loans (EIDL)
☐ Small Business Administration (SBA)Loan Forgiveness
☐ Other Federal programs
☐ This business has not received financial assistance from any Federal program since March 13, 2020
Details:
The cognitive testing staff propose the improvements to the response categories. These changes largely focus on capitalization and error correction.
Original text:
What is this establishment’s 9-digit Employer Identification number (EIN) used on its latest Internal Revenue Service 941 Employer's Federal Quarterly Tax Return?
Proposed text:
What is this business’s 9-digit Employer Identification number (EIN) used on its latest Internal Revenue Service 941 Employer's Federal Quarterly Tax Return?
Details:
The cognitive testing staff recommend changing the term establishment to business to maintain consistency within the instrument.
The following is the updated questionnaire including the proposed changes requested by the Census Bureau for approval for collection with week 4.
Overall, how has this business been affected by the COVID-19 pandemic?
Select only one:
☐ Large negative effect
☐ Moderate negative effect
☐ Little or no effect
☐ Moderate positive effect
☐ Large positive effect
In the last week, did this business experience a change in operating revenues/sales/receipts, not including any financial assistance or loans?
Select only one:
☐ Yes, increased
☐ Yes, decreased
☐ No
In the last month, what were the total operating revenues/sales/receipts for this business, not including any financial assistance or loans?
Select only one:
☐ $0 - $500
☐ $501 - $2,500
☐ $2,501 - $5,000
☐ $5,001 - $15,000
☐ $15,001 - $50,000
☐ $50,001 - $125,000
☐ $125,001 - $200,000
☐ $200,001 - $500,000
☐ $500,001 or more
☐ Don’t know
In the last week, did this business temporarily close any of its locations for at least one day?
☐ Yes
☐No
In the last week, did this business have a change in the number of paid employees?
Select only one:
☐ Yes, increased
☐ Yes, decreased
☐ No
In the last week, did this business have a change in the total number of hours worked by paid employees?
Select only one:
☐ Yes, increased
☐ Yes, decreased
☐ No
In the last week, did this business have disruptions in its supply chain?
☐ Yes
☐No
In the last week, did this business shift to the production of other goods or services?
☐ Yes
☐No
In the last week, did any of the business’s locations adopt pickup/carry-out/delivery as their only means of providing goods and services to their customers?
☐ Yes
☐ No
How would you describe the current availability of cash on hand for this business, including any financial assistance or loans? Currently, cash on hand will cover:
Select only one:
☐ 1-7 days of business operations
☐ 1-2 weeks of business operations
☐ 3-4 weeks of business operations
☐ 1-2 months of business operations
☐ 3 or more months of business operations
☐ No cash available for business operations
☐ Don’t know
The White House declared a national emergency because of the COVID-19 pandemic on March 13, 2020. Since then, has this business missed any loan payments? Loan payments that have been forgiven or postponed should not be considered to be missed.
☐Yes
☐No
Since March 13, 2020, has this business missed any other scheduled payments, not including loans? Examples of other scheduled payments include rent, utilities, and payroll. Scheduled payments that have been forgiven or postponed should not be considered to be missed.
☐Yes
☐No
Since March 13, 2020, has this business requested financial assistance from any of the following sources?
Select all that apply:
☐ Paycheck Protection Program (PPP)
☐ Economic Injury Disaster Loans (EIDL)
☐ SBA Loan Forgiveness
☐ Other Federal programs
☐ State or local government programs
☐ Banks
☐ Self
☐ Family or friends
☐ Other sources
☐ This business has not requested financial assistance from any source since March 13, 2020
Since March 13, 2020, has this business received financial assistance from any of these programs from the Federal government?
Select all that apply:
☐ Paycheck Protection Program (PPP)
☐ Economic Injury Disaster Loans (EIDL)
☐ SBA Loan Forgiveness
☐ Other Federal programs
☐ This business has not received financial assistance from any Federal program since March 13, 2020
In your opinion, how much time do you think will pass before this business returns to its normal level of operations relative to one year ago?
Select only one:
☐ 1 month or less
☐ 2-3 months
☐ 4-6 months
☐ More than 6 months
☐ I do not believe this business will return to its normal level of operations relative to one year ago
☐There has been little or no effect on this business’s normal level of operations relative to one year ago
What is this business’s 9-digit Employer Identification number (EIN) used on its latest Internal Revenue Service 941 Employer's Federal Quarterly Tax Return?
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Catherine D Buffington (CENSUS/ADEP FED) |
| File Modified | 0000-00-00 |
| File Created | 2021-01-14 |