OMB Memo

Generic Information Collection Request:
Cognitive Testing for the 2025 Annual Business Survey

Request: The Census Bureau plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). The Census Bureau plans to conduct pretesting activities for the 2025 Annual Business Survey (ABS).

The ABS is conducted annually as a supplement to the economic census collected by the Census Bureau and sponsored by the National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation (NSF).

The ABS provides a detailed, regularly collected source of data on the status, nature, and scope of women-, minority-, and veteran-owned businesses. The ABS provides estimates for the number of employer firms, sales and receipts, annual payroll, and employment by owners’ gender, ethnicity, race, and veteran status. The ABS also provides information on research and development activity and cost for businesses with 1-9 employees and nonprofit organizations and includes survey questions to measure business innovation in firms of all sizes. For the 2024 ABS, estimates will be available for the U.S., states, and select metropolitan statistical areas (MSA). Among other uses, the Small Business Administration (SBA) and the Minority Business Development Agency (MBDA) use results from the ABS to assess business assistance needs and allocate available program resources. Data collected on research and development and innovation may be used to compare R&D costs across industries, determine where R&D activity is conducted geographically, and identify the types of businesses with R&D, to contribute to the Bureau of Economic Analysis (BEA) system of national accounts; to increase investments in research and development, strengthen education, and encourage entrepreneurship; and to compare US business innovation with innovation in other countries, including those in the European Union.

The ABS is primarily collected via a self-administered questionnaire using the Census Bureau’s online survey reporting system. Respondents are mailed a letter informing them of the requirement to complete the survey and providing them with access information. A preview of the questionnaire is available for respondents to download to review the survey questions and instructions. For more information regarding ABS: https://www.census.gov/programs-surveys/abs.html

Purpose: The ABS asks questions about the business owner(s) as well as various characteristics about the business. The 2025 ABS cognitive testing will include new potential questions. The topics of questions are listed below:

• Cybersecurity – seeks to understand the industry’s definition, organization, and composition of employed cybersecurity workers as well as the cost breakdown of services.

• Innovation and Technology – asks about the barriers to conducting innovation activities as well as the types of business strategies used to improve or develop goods or products; seeks to understand businesses’ engagement with various technologies and the role that different technologies play.

• Nonprofit – testing will be done on a reorganization of the questionnaire including revised questions and terminology changes.

Results: The results from the cognitive testing will be documented in a report, outlining the findings of the pretest and suggesting recommendations for improvement to questions. This report will be shared with OMB ahead of the proposed information collection request for the 2025 ABS. This report will include:

• Understanding how respondents comprehend specific questions

• Identifying respondents’ use of records and/or estimation strategies for answering specific questions

• Assessing respondents’ ability to answer specific questions

• Identifying difficulties in completing the questionnaire (where applicable)

• Recommended changes to questions, or the addition of new questions, and response options to be implemented in the 2025 ABS

Population of Interest: Sample members from the 2020, 2021, 2022, and 2023 ABS. Businesses should be representative of all business sizes and cover a variety of industries. More details about the sample can be found under the Sample section below.

Timeline: Testing will be conducted from June through September 2024.

Language: Testing will be conducted in English only.

Method:

Cybersecurity, Innovation, and Technology (CIT) testing:

We will conduct cognitive interviews with up to 45 respondents across three rounds of iterative testing in order to evaluate and refine the new and revised questions. These interviews will be conducted and recorded via Microsoft Teams. The first round will include targeted interviewing of respondents from companies with cybersecurity expertise using a guided protocol (Attachement B1) to help with the development of questions to be tested in the second round. For the rest of the CIT portion of the testing, the next rounds will use a draft of the proposed 2025 ABS questions (See Attachment A, new questions and text are highlighted in yellow) as well as probes (see Attachment B2) to assess respondents’ understanding of the questions and the flow of the questionnaire. The development of more questions to be tested in the second and third rounds will take place after the previous round has concluded and will be informed by findings from the first round. Interviews will be conducted by researchers from the Data Collection Methodology and Research Branch (DCMRB) and Special Sworn Status staff from the NCSES. Staff from NCSES and/or Economic Reimbursable Division may observe if available.

In later rounds, we will also conduct online unmoderated cognitive testing via the Qualtrics platform with up to 200 respondents total. These respondents will be asked to fill out a limited number of probes from the cognitive interviewing protocol (see Attachments A and B2).

Nonprofit (NP) Module testing:

We will conduct cognitive interviews with up to 20 respondents in order to evaluate and refine the new and revised questions. These interviews will be conducted and recorded via Microsoft Teams. A blank draft of the proposed 2025 Nonprofit module questions will be emailed to scheduled respondents prior to the interviews (Attachment D). During the interviews, respondents will view the proposed questions. Interviews will be conducted by researchers from DCMRB and NCSES. Special Sworn Status staff from the NCSES and/or Economic Reimbursable Division may observe if available. We will use probes (see Attachment C) to assess respondents’ understanding of the questions and the flow of the questionnaire.

Confidentiality: All participants will be informed that their response is voluntary and that the information they provide is confidential and will be seen only by Census Bureau employees and those with special sworn status (see Attachment E).

Sample:

Cybersecurity, Innovation, and Technology

We plan to conduct 45 cognitive interviews over three rounds of cognitive testing. Half the sample will consist of companies in the Professional, Scientific, and Technical Services NAICS industries, and the other half will be randomly sampled. One third of the entire sample will consist of companies with over 1000 employees, another third with fewer than 50 employees, and the last third will consist of companies with 50-1000 employees. Sampling for the targeted first round will include companies with cybersecurity expertise who are in the Business Register. In the other rounds, half of the the companies will also have either a cybersecurity department or program in-house or outsource their cybersecurity work, and the other half will be randomly sampled. The latter sampling strategy will also be used in unmoderated testing.

Nonprofit Module testing:

We plan to conduct a maximum of 20 cognitive interviews with an equal distribution between large, medium, and small nonprofits as well as an equal distribution between health vs. non health care organizations.

Recruitment: Participants will be recruited via email. Once interviews are scheduled, researchers will send participants a confirmation email. Before beginning the interviews, researchers will inform participants that their response is voluntary and that the information they provide is confidential under Title 13. Researchers will confirm that the respondents have read, understand, and agree to the contents of the consent form (see Attachment E).

Protocol: Copies of the draft interview protocols and draft questionnaires for testing purposes are enclosed (see Attachments A - D).

Cognitive interviews for CIT and NP will be conducted via Microsoft Teams. Participants will be asked to read the draft questions. Probes will be asked concurrently. We anticipate the CIT sessions will take 45 minutes to complete, and 30 minutes for the NP sessions. Only one participant will be interviewed at a time.

Use of incentive: Monetary incentives for participation will not be offered.

Enclosures: Below is a list of materials to be used in the current study:

• Attachment A: 2025 ABS Cybersecurity, Innovation and Technology Module Draft

• Attachment B1: 2025 ABS Cybersecurity Cognitive Testing Protocol

• Attachment B2: 2025 ABS Innovation and Technology Cognitive Testing Protocol

• Attachment C: 2025 ABS Nonprofit Module Cognitive Testing Protocol

• Attachment D: 2025 ABS Nonprofit Research Activities Module Draft

• Attachment E: Consent Form

Burden Estimate: We expect that each Cybersecurity, Innovation, and Technology cognitive interview will last no more than 45 minutes (45 cases x 45 minutes per case = 33.75 hours). The nonprofit interviews will last no more than 30 minutes (20 cases X 30 minutes per case = 10 hours).

Additionally, to recruit respondents testing, we expect to make up to 5 email contacts per completed case. The recruiting emails are expected to take on average 3 minutes to read [(5 attempted emails per completed case x 65 cases x 3 minutes per case) / 60 minutes] = 16.25 hours).

The online testing instrument will be administered to 200 respondents and will take up to 15 minutes to complete. This results in a total burden of 50 hours for online testing.

Thus, the total estimated burden for the cognitive interview portion of this project is 110 hours (43.75 hours for interviews + 16.25 hours for recruiting + 50 hours for unmoderated testing ).

Contact: The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Hillary Steinberg

Data Collection Methodology and Research Branch/Economic Statistical Methods Division

U.S. Census Bureau

Washington, D.C. 20233

(301) 763-5383

[email protected]

cc.:

Nick Orsini (ADEP) with enclosures

Stephanie Studds (ADEP) with enclosures

Audrey Kindlon (NSF) with enclosures

Kevin Deardorf (ERD) with enclosures

Aneta Erdie (ERD) with enclosures

Patrice Hall (ERD) with enclosures

Katie Genadek (ESMD) with enclosures

James Hunt (ESMD) with enclosures

Temika Holland (ESMD) with enclosures

Rebecca Keegan (ESMD) with enclosures

Aleia Clark Fobia (ADRM) with enclosures

Jasmine Luck (ADRM) with enclosures

Danielle Norman (PCO) with enclosures

Mary Lenaiyasa (PCO) with enclosures