SBPS Phase 4 Cognitive Pretesting Nonsub

Nonsubstantive Change Request:
Cognitive Interviewing for Phase 4 of the Small Business Pulse Survey (SBPS)

Request: The Census Bureau plans to conduct additional research under the Small Business Pulse Survey clearance (OMB number 0607-1014). We will be conducting cognitive interviews by phone lasting no longer than 45 minutes each to test newly developed and revised questions about the effect of changing business conditions due to the Coronavirus pandemic and other social changes on small businesses in the United States.

In response to the rapid changes in the economic landscape, the U.S. Census Bureau has conducted the Small Business Pulse Survey weekly to measure the effect of changing business conditions due to the Coronavirus pandemic on small businesses. This weekly survey focused on the changes in small businesses’ incoming revenue and outgoing expenses, staffing, and business models. The questions proposed for cognitive testing at this time cover the following topics:

• Requiring COVID-19 testing

• Requiring COVID-19 proof of vaccination

• Decrease in leased/rented space

• Adding pickup/carryout/delivery

• Changes to capital expenditures

• Plans for business travel expenditures

As conditions for small businesses continue to unfold, the Census Bureau continues to engage with stakeholders to understand what data remain relevant and what data are needed. In consultation with those stakeholders, the Census Bureau proposes to test the content identified in this request for potential implementation in future cycles of data collection of the SBPS. Researchers from the Census Bureau’s Economic Statistcal Methods Division (ESMD) will conduct a series of 45 minute phone-based cognitive interviews with small businesses. Given time and resource limitations, we are not using Skype for Business or video for these interviews.

Purpose: The purpose of cognitively testing newly proposed questions for the SBPS is to ensure that the questions are measuring the underlying constructs of interest and to better understand the accessibility of the requested data and the burden of compiling responses to the questions. The feedback from these interviews will be used to refine question wording and decide whether (or not) to include these new questions in future cycles of data collection. Revisions to previous questions are also being tested.

Population of Interest: Representatives of small businesses across all sectors. These businesses will meet at least one of the following criteria (as determined from their responses to the 2017 Economic Census), in accordance with eligibility parameters for the PPP¹:

• Any small business concern that meets SBA’s size standards (either the industry based sized standard or the alternative size standard);

• Any business, 501(c)(3) non-profit organization, 501(c)(19) veterans organization, or Tribal business concern (sec. 31(b)(2)(C) of the Small Business Act) with the greater of:

  • 500 employees, or

  • That meets the SBA industry size standard if more than 500

• Any business with a NAICS Code that begins with 72 (Accommodations and Food Services) that has more than one physical location and employs less than 500 per location; and/or

• Sole proprietors, independent contractors, and self-employed persons

Timeline: Testing will be conducted during January, 2021

Language: Testing will be conducted in English only.

Method: The method of research will be cognitive interviewing. The purpose of cognitively testing the proposed set of SBPS questions is to minimize measurement error and maximize the validity of these questions by ensuring that the questions accurately measure the underlying construct of interest. Cognitive interviewing is a method of pre-testing instruments that involves in-depth interviewing, paying particular attention to the mental processes respondents use to respond to questions². Cognitive interviewing uses a framework dependent on evaluating questions against their outcome objectives, including accurately eliciting the underlying construct of interest, and to what level of accuracy respondents can provide data in response

All interviews will be conducted over the telephone. The interviewers will follow a semi-structured interview protocol (Attachment A). Interviewers will send a link to the survey online to respondents prior to calling so that respondents can work through the survey while on the phone. The survey will be hosted through the Qualtrics online survey platform. Interviews will not be recorded.

Sample: We plan to conduct a maximum of 40 interviews total over two rounds of data collection. In the first round, we will target 20 to 25 respondents across all industries. The research staff will then review the resultant data and make adjustments to the interviewing protocol and to the questions if necessary. We will then engage a second round of interviewing targeting 10 to 15 respondents, again across all industries. This number of interviews is targeted because it is a manageable number of interviews for the time period allotted, it should adequately cover targeted small businesses, and it should be large enough to provide reactions to the questions in order to identify meaningful findings. Table 1 outlines the recruitment matrix that will guide this work; note that the matrix is a target, and while we will attempt to recruit across sectors, we do not expect the final frequency distribution to match below.

Sector	Round 1 Recruitment Goal	Round 2 Recruitment Goal	Total
Mining	4	2	6
Service	10	6	16
Construction	3	2	5
Manufacturing	2	1	3
Wholesale	3	2	5
Retail	3	2	5
Total	25	15	40

For this research, we will target sample from the Phase 1 SBPS from weeks 5, 6, and 7. We have specifically chosen these respondents because their last contact with the Census Bureau was long enough ago to minimize satisficing due to familiarity with the questions, but recent enough that the research is still salient. We will include both respondents to the first round of the SBPS, and non-respondents, if they agree to participate. Additionally, we will augment the sample with small businesses that responded to the 2017 Economic Census in sectors of interest; these are firms that have direct contact with the public, and so, we want to ensure multiple interviews within these sectors given the subject matter of the new questions.

Recruitment: Participants will be recruited using a sample file from the Phase 1 of SBPS and 2017 Economic Census as described above. First, we will send an email to the contact information listed in the 2017 Economic Census data and prior SBPS sample files. This email will include instructions for respondents to schedule an interview time and date with a Census Bureau researcher.³ We will verify the appointment time, and respond by email to the respondent with confirmation of scheduling; in that email, we will also verify the best number to reach the respondent. About 30 minutes before their appointment time, we will email the respondent again reminding of the upcoming appointment and including a link to the survey. The first screen of the online survey will be a Paperwork Reduction Action (PRA) and Privacy Act (PA) statement, informing participants that their response is voluntary and that the information they provide is confidential under Title 13 and asking for consent to be interviewed. Respondents will need to click a checkbox indicating that they understand these rights and agree to be interviewed. If email recruitment is not meeting our recruitment goals, we will move to telephone calls. We may schedule a few additional interviews, in case of no-shows. These will be canceled if not needed.

Protocol: A copy of a draft interview protocol is enclosed. Respondents will answer all questions in the survey, but we will only probe in detail on the new or revised questions.

Use of incentive: Monetary incentives for participation will not be offered.

Below is a list of materials to be used in the current study:

Attachment A: Draft protocol outlining intended questions to guide the interviews

Attachment B: Draft instrument, including PRA/PA statements

Length of interview: For cognitive interviews, we expect that each interview will last no more than 45 minutes (40 cases x 45 minutes per case = 30 hours). Additionally, to recruit respondents we expect to reach out via email and to make up to 3 phone contacts per completed case. The recruiting emails and calls are expected to last on average 3 minutes per call (3 attempts per phone call per completed case x 40 cases x 3 minutes per case = 6 hours). Thus, the estimated burden is 36 hours (30 hours for interviews + 6 hours for recruiting). There are no plans for observers for these interviews.

The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Melissa A. Cidade, Ph.D.

Data Collection Methodology and Research Branch

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-8325

[email protected]

cc:
Nick Orsini (ADEP) with enclosures

Cathy Buffington (ADEP)     “          ”

Michael Lopez Pelliccia (ADEP)     “          ”

Lucia Foster (CES)     “          ”

Carol Caldwell             (ESMD)     “          ”
Amy Anderson Riemer  (ESMD)     “          ”

Diane Willimack (ESMD)     “          ”

1 See United States Small Business Administration. “Paycheck Protection Program.” Retrieved from: https://www.sba.gov/funding-programs/loans/coronavirus-relief-options/paycheck-protection-program, June 16, 2020.

2 Campanelli, P. 2007. “Methods for Testing Survey Instruments.” Short Course, Joint Program in Survey Methodology (JPSM). Arlington, VA.

3 Note: If we are not meeting recruitment goals using email contact, we will switch to telephone calls to increase participation.