2022 BRS Pilot Test and Debriefing Report

FROM: Douglas Williams, Robin Kaplan

Office of Survey Methods Research

SUBJECT: Debriefing Results for the Business Response Survey 3.0 (DRAFT)

DATE: July 13, 2022

Introduction

The Bureau of Labor Statistics (BLS) developed the Business Response Survey (BRS) as a tool allowing for the measure of new and emerging changes in the U.S. economy. The BRS is conducted online using the BLS Internet Data Collection Facility (IDCF) as a follow-on survey to the Annual Refiling Survey (ARS). Pilot tests of the BRS were initially conducted in 2018 and 2019 with the first full-production implementation conducted in 2020. The data collection conducted in 2020 as a stand alone survey, focused on the effect of the coronavirus pandemic on businesses. This was followed by the first follow-on BRS data collection in 2021 that measured how businesses continued to operate through the peak of the coronavirus pandemic.

The BRS is conducting another data collection for 2022, with a focus on telework, hiring, and vacancies. Some questions on telework will provide updated information from the previous BRS, while content on hiring and vacancies is new. Prior to the production fielding, a small pilot test was conducted with a select number contacted for debriefing interviews. This memo provides results on the pilot test and debriefing interviews.

Method

The production version of the BRS was developed through a team of BLS content experts, research statisticians, and survey methodologists. This ensured the question topics were relevant to the purpose of the BRS while considering the resulting data and potential cognitive issues. The initial questionnaire that resulted was reviewed to address questionnaire wording issues, ensure consistency in terminology use, and reference periods.

Following the questionnaire development and review, 2,001 cases were selected for a small pilot test. The pilot test simulated the production environment, where sampled establishments were invited via email to complete the BRS online. The first email was sent on June 7, 2022, with a follow up on June 21, 2022.

Establishments completing the online questionnaire for the pilot test were reviewed to identify establishments to invite to a debriefing interview. The 2022 BRS asks about telework, hiring, and vacancies. Many businesses may not have any activity to report for these topics (e.g., no use of telework, no hiring, or no vacancies during specified reference periods). Key questions were used to select establishments reporting some of these activities to ensure coverage of as many questions in the BRS as possible. The questions used for initial review are listed below.

• Q1 – Do any employees at this location CURRENTLY telework in any amount?

• Q5 – In April 2022, did this location hire any new employees?

• Q9 – Were any of the newly hired employees in April 2022, for positions open for MORE THAN 30 days?

• Q16 – Does this location currently have any open positions that you are looking to fill?

Question 5 was prioritized since a “no” response would result in the most questions skipped. Efforts were made to ensure coverage of as many questions as possible. Due to differential participation by sampled establishments and differences in reports for these activities, the number of debriefing participants for some questions will vary. Any respondents who did not hire in April of 2022, were asked general probes to get at their comprehension of the hiring questions (e.g., how easy or difficult it would be to answer on a future survey, and how they would arrive at their answers).

A debriefing protocol was developed to outline potential issues and serve as a guide for interviewers. Interviewers could deviate from the protocol and probe as needed to address issues that may not be included in the protocol.

Participants

The pilot test sample was selected from establishments where BLS had contact information for a person at the establishment. This required at least an email address, but most cases also included a name, and telephone number. Participants for debriefing interviews were selected from those completing the pilot test, with priority given to those with specific response patterns as discussed in the section of this report on methods. Table 1 below shows the number and percent of emails that were undeliverable, the total complete responses received, and the number of incomplete responses. A total of 523 establishments completed the pilot test for a completion rate of 26.1 percent.

Table 1. Number and percentage of emails and responses that were undeliverable, completes, and incompletes (N=2,001)

Undeliverable	Complete Responses	Incomplete
54	523	17
2.7%	26.1%	2.6% of responses

Select pilot test participants that completed the survey were invited via email to participate in debriefing interviews with some telephone follow-up as needed. Cases that were invited were informed the debriefing would last approximately 45-60 minutes. After a completing the first initial debriefing interviews, this was revised down to 30-45 minutes, reflecting the actual time interviews were taking. This revision also helped to minimize the perceived burden in the invitation. A total of 233 pilot test participants were invited to a debriefing interview, with 16 completing an interview. The establishment characteristics of these participants and response to key questions for selection are shown in Table 2.

Table 2. Establishment characteristics of debriefing participants.

Industry	# of Employees at location*	Q1 Telework*	Q5 Hiring*	Q9 Hired 30+*	Q16 Vacancies*
541690 - Other Scientific and Technical Consulting Services	35	X	X		X
721110 - Hotels (except Casino Hotels) and Motels	23		X	X	X
327390 - Other Concrete Product Manufacturing	9		X	X
448150 - Clothing Accessories Stores	Missing		X		X
333249 - Other Industrial Machinery Manufacturing	43		X	X	X
721110 - Hotels (except Casino Hotels) and Motels	15 to 25		X		X
722515 - Snack and Nonalcoholic Beverage Bars	10		X	X	X
446110 - Pharmacies and Drug Stores	42		X	X
311412 - Frozen Specialty Food Manufacturing	600	X	X	X	X
722515 - Snack and Nonalcoholic Beverage Bars	18	X	X
332510 - Hardware Manufacturing	41		X		X
522130 - Credit Unions	139	X	X	X
519190 - All Other Information Services	135	X	X	X	X
541340 - Drafting Services	13	X			X
522130 - Credit Unions	260	X			X
315280 - Other Cut and Sew Apparel Manufacturing	48		X

* # of Employees at location indicates the number of employees the respondent provided for the sampled location during the debriefing interviews; Q1 Telework indicates employees telework for the establishment; Q5 Hiring indicates employees were hired in April 2022; Q9 Hired 30+ indicates that some employees hired in April 2022, were for position open for more than 30 days; and Q16 Vacancies indicate the establishment has current vacancies.

Results

The pilot test approach provided a way to evaluate questionnaire responses in a setting simulating a production environment – that is, matching how respondents will interact with the survey. Pilot test data was reviewed to identify problem indicators, such as item nonresponse, or inconsistent reporting for inter-related questions. The pilot test also included survey quality measures such as difficulty completing the survey, sources of difficulty, and perceived relevance of the survey.

General Findings

The survey took an average completion time of 4.5 minutes, with a range of 1 minute to 25 minutes. A review by BLS staff indicated the overall response patterns made sense and were in line with expectations. The “soft” skip patterns with instructions-only to “Skip to question X” worked well, but some respondents completed questions that they could have skipped. Respondents completed these questions correctly (e.g., they have no telework and should skip the telework percentages, but they fill that in as 100% No Telework). More intuitive skip pattern visuals are planned in the production version of the survey.

Respondents also answered questions correctly overall in terms of the numeric responses (e.g., Overall hires > Hires that took <30 days). However, in looking at a ratio of average monthly employment to the number of new hires reported in April, most respondents reported values that were in line with expectations, but about 1% of respondents reported values that were higher than expected. One reason for this could be that they are providing a firm-level response. Thus, a line will be added to the question reiterating “this location only, not including other locations of this company.”

The questions that asked about professional license or bachelor’s degree requirements for new hires that took both >30 and <30 days to hire gave respondents 3 options to select: All, Some or None. Initial results showed that most respondents with >5 hires are selecting “Some” for both >30 and <30 days, yielding results that will not be meaningful in making comparisons or distinctions. These questions were updated to ask for approximate percentages, rather than the 3-level Likert scale options. The follow up email blast tested respondent ability to provide approximate percentages.

Individual Question Response Rates

Individual question response rates ranged from 95-99%. This low item nonresponse rate suggests that respondents were willing and able to answer most questions on the pilot survey. A total of 86.2% respondents indicated the survey was at least somewhat relevant for understanding current change in the labor market. Table 3 shows the percent of respondents that found the survey easy or difficult to complete, with the majority finding it very easy or somewhat easy.

Table 3. Respondent ratings of how easy or difficult it was to complete the survey (N = 2,001)

Very easy	63.97%
Somewhat easy	22.35%
Neither easy nor difficult	9.44%
Somewhat difficult	3.47%
Very difficult	0.77%

Additionally, as seen in Table 4, the minority of respondents who found the survey difficult to complete indicated the top reason was due to having to consult company records. Note that this response rate did not differ from previous production, suggesting that the survey was not so burdensome as to affect overall response.

Table 4. Respondents’ reasons for finding the survey difficult to complete (n = 85)

Consult records	31.91%
Consult with other company employees	14.89%
Questions required calculations	9.57%
Some questions were not clear	21.28%
Some other reason	22.34%

General Respondent Feedback

During the debriefing interviews, many respondents expressed recent difficulties in identifying applicants, hiring, or retaining new hires, and how the COVID-19 pandemic may have affected these issues. Due to this, a new probe was added to the debriefing protocol to allow respondents to provide open-ended comments on this, which respondents appreciated. This reinforces the need to add an optional open-ended item at the end of the survey where respondents can provide this information.

Most respondents identified themselves as the best person to answer these types of questions in the future. They were usually company owners, managers, CFOs, or HR staff. Respondents mentioned the questions that required them to look at records included the items on hiring (e.g., how many people were hired in the reference month), how long the positions had been open for, what percent of staff telework or work remotely, or educational requirements for positions. They usually looked at company records, including payroll, timekeeping records, recruitment software, or checked with other staff members to verify these answers.

Question by Question Findings

The following section outlines the main findings for each BRS question (as well as instructions and transition statements) based on feedback from the 16 debriefing interviews.

Instructions

The Bureau of Labor Statistics (BLS) is looking to understand telework, hiring, and vacancies at your business. This information is particularly critical in understanding how the labor market has changed because of the COVID-19 pandemic. You’ll see questions about telework at your location both currently and before the pandemic, and questions about newly hired workers and existing vacancies at your business location.

Please answer the following questions for this business location only.

Finding: Respondents thought the introduction was clear and easy to understand, and most had no problems determining which location to report for. One respondent who owned a company where the employees were all 100% said that the language about “this location” did not apply to them since the business does not have a physical location. He answered instead about the company as a whole.

Recommendation: No changes are recommended.

Question 1

Telework is a work arrangement that allows an employee to work at home, or from another remote location, by using the internet or a computer linked to one’s place of employment, as well as digital communications such as email and phone.

1. Do any employees at this location CURRENTLY telework in any amount?

• Yes

• No -> GO TO Q3

Finding: Respondents generally understood the question about whether the location has employees who currently telework in any amount. A few respondents displayed minor issues with the term “telework.” For example, one respondent thought the term was related to telemarketing only and another felt the term is outdated and favored “remote” work. Several respondents who worked at businesses that require employees to be on-site (e.g., retail stores, banks, manufacturing businesses) wavered a bit on whether to respond “yes” or “no” to this question because some employees would occasionally catch-up on emails or phone calls offsite, but most ultimately answered “no.” These were usually specific types of staff (e.g., IT professionals, accountants, sales people) doing tasks such as payroll, responding to emails, or billing. For a couple respondents, this type of telework is not formally recorded or required by the company, so they weren’t sure if it met the definition. Two respondents noted they allow ad-hoc telework if employees do not feel well, but that this is rare.

Adding the phrase “telework or work from home” to the question text was considered to help respondents better understand the meaning, but ultimately no question changes were recommended. As noted, the telework description includes “work from home” and telework is typically understood to be “paid work,” as in employees are paid to work from an alternate location. It’s possible that by including ‘work from home’ in the body of the question we’d be deviating from the paid work concept and would unnecessarily open up the telework questions to be answered affirmatively, for every employee that might work 9-5 in the office and occasionally check their email from home.

Recommendation: No question changes are recommended.

Question 2

2. In a typical week, what percent of employees CURRENTLY telework in the following amounts?

Answers should total 100%

____ all the time (remote employee)

____ some of the time (some work hours or days via telework)

____ rarely or never (rare occasions of telework, or full-time on-site)

Finding: The question on what percent of employees currently telework all the time, some of the time, or rarely/never was clear and well understood. Most respondents could easily pull this information from company records, or did some quick calculations to determine what percent was appropriate for each category. A couple respondents noted they do not formally track this information in company records, so they inferred by job titles and positions at the location and who typically teleworks or comes on-site.

Recommendation: No question changes are recommended.

Question 3

3. In the next 6 months, does this location expect the amount of time that employees are permitted to telework to…

• increase

• decrease, or

• stay the same?

Finding: The question about whether telework would increase, decrease, or stay the same in the next 6 months was well-understood and clear to respondents. Most respondents found this question easy to answer since they did not expect the current policy to change or because they would be gradually phasing employees to on-site work in the next 6 months.

Recommendation: No question changes are recommended.

Question 4

4. In February 2020, before the coronavirus pandemic began, did any employees at this location telework in any amount?

• Yes

• No

• Don’t know

• Location not in business in February 2020

Finding: The question about whether employees teleworked before the pandemic began was generally well-understood by respondents and easy to answer since several had expanded telework during the pandemic. For businesses where telework was not possible, the question felt a little redundant to them, but they were able to provide an answer easily.

Recommendation: No question changes are recommended.

Question 5

5. In April 2022, did this location hire any new employees?

Include employees that were hired, even if they have not yet formally started.

• Yes

• No -> GO TO Q16

Finding: The question on whether respondents hired any new employees in April of 2022 was generally well understood and clear to respondents. Most respondents reported that they used records to answer the question, but some knew off the top of their head. Most respondents understood the italicized instructions to “include employees that were hired, even if they have not yet formally started” to mean that job candidates were extended a job offer but their start date had not yet come, or they were not officially on payroll yet. A few respondents remarked that only employees that actually started at the job should be included because a lot of new hires “ghost them,” or just never show up on their first day. At least two respondents noted they did not include people who were hired in April of 2022, but did not show up on their first day of work.

Additionally, a few respondents remarked on the short reference period or “snapshot” of just April 2022; they felt focusing only on hiring during this month was not necessarily representative of the hiring the company does throughout the year. One participant suggested a three-month window would have been more representative. At least two respondents also seemed to telescope their responses – bringing in hires that occurred just before April of 2022. This type of issue may occur more at smaller or medium sized establishments where the respondent may use memory to arrive at the answer rather than company records.

Recommendation: Add the following text to the question instructions, “Include employees that were hired, even if they have not yet formally started or have left the position since being hired.” This instruction would help respondents know we want to collect all hires, even if they didn’t show up on their first day or left the position shortly after being hired.

Question 6

5. In April 2022, how many new employees did this location hire?

[Numeric text box]

Finding: The question on how many new employees the location hired was well understood and clear to respondents. Most respondents checked payroll records or knew this information since they are involved in hiring. We also looked at a ratio of average monthly employment to the number of new hires reported in April. Most respondents reported values that were in line with expectation. About 1% of respondents reported values that were higher than expected. One reason for this could be that they are providing a firm-level responses. Thus, reiterating in the instructions to include “this location only, not including other locations of this company” is recommended.

Recommendation: Add the following language to the section and question instructions, “Please include only new hires at this specific location, and not any other locations of this company.”

Question 7

5. How many of these new employees hired in April 2022, will telework all the time (be remote employees)?

[Numeric text box]

Finding: The question on how many new employees will telework all the time was clear and well understood. Most respondents checked payroll records or knew this information since they are involved in hiring.

Recommendation: No question changes are recommended.

Question 8

5. For positions filled in April 2022, did this location do any of the following to attract more applicants?

Select all that apply.

• Expanded advertising

• Started using recruiters/talent agencies

• Increased starting pay

• Offered hiring bonuses

• Offered more hours (e.g., changed position from part-time to full-time)

• Reduced qualifications (e.g., education or experience)

• Expanded benefits

• None of the above

Finding: The question on ways the business location tried to attract more applicants was generally well understood and clear to respondents. A few respondents mentioned it would have been helpful to have an Other, specify option to indicate other ways they attracted candidates. Some of these other ways included working with a bridge refugee center, attending career fairs at community colleges, working with a prison ministry, dropping drug test requirements, delaying background checks, holding company team building events, industry networking, marketing, use of employee referrals, increasing training/internship opportunities for new hires, or allowing more telework. Ultimately, including an additional open-ended response option was determined to be too burdensome and not recommended. However, a few respondents mentioned that employees want expanded telework opportunities and that this might be a response option to consider adding. Because this is one of the main topics of the BRS, adding this response option is recommended.

Recommendation: Add the response option “Expanded telework or remote work.”

Question 9

5. Were any of the newly hired employees in April 2022 for positions open for MORE THAN 30 days?

• Yes

• No -> GO TO Q14

Question 10

5. How many of the newly hired employees in April 2022 were for positions open for MORE THAN 30 days?

[Numeric text box]

Finding: Questions 9 and 10 on whether any of the newly hired employees in April 2022 were for positions open more than 30 days (and how many), were both well understood and clear to respondents. However, there was some ambiguity in these questions for a few respondents. For example, several businesses had a high level or turnover and are constantly recruiting more employees, so it was hard to say whether any one particular position had been open more than 30 days, or whether they were part of the same posting or replacement postings for people who recently quit or did not show up on their first day. Two respondents noted that the positions at their companies are “fungible” in that if they interview a candidate for an open position but realize their skillset is better suited to a different area, they might create a new job for that individual. Another respondent noted that for positions that are highly skilled, it is often difficult to find qualified candidates and so they are “always on the lookout” to fill those positions. Because respondents understood the question well, no changes are recommended, but we note these potential sources of measurement error.

Recommendation: No question changes are recommended.

Question 11

5. For positions that took MORE THAN 30 days to fill, how many of these required a professional, state, or industry license or certificate?

• All

• Some

• None

• Don’t know

(Revised version) For positions that took MORE THAN 30 days to fill, about what percent of these required a professional, state, or industry license or certificate?

Enter approximate percent (E.g., 10 = 10%)

[Numeric text box]

Finding: The question on positions that took more than 30 days to fill requiring a license or certificate was generally well understood and clear to respondents. The question was easy for most respondents to answer, examples included engineering certificates, food handling certificates, and a mortgage license. Several respondents remarked that some of these licenses or certificates are preferred or beneficial, but not required. As noted above in the General Findings section, a revised version of this question was later tested that asked for percentages instead of Likert scale options, as the Likert scale was not yielding meaningful distinctions. These responses to the updated online questionnaire revealed no difficulty in respondents being able to provide approximate percentages.

Recommendation: Use the revised version of the question asking for percentages.

Question 12

5. For positions that took MORE THAN 30 days to fill, how many of these required a bachelor’s degree (4-year degree, BA or BS) or higher?

• All

• Some

• None

• Don’t know

(Revised version) For positions that took MORE THAN 30 days to fill, about what percent of these required a bachelor’s degree (4-year degree, BA or BS) or higher?

Enter approximate percent (E.g., 10 = 10%)

[Numeric text box]

Finding: The question on positions that took more than 30 days to fill requiring a BA degree or higher was generally well understood and clear to respondents. Several respondents remarked this level of education is preferred, but not a requirement. As noted above in the General Findings section, a revised version of this question was later tested that asked for percentages instead of Likert scale options, as the Likert scale was not yielding meaningful distinctions. These responses to the updated online questionnaire revealed no difficulty in respondents being able to provide approximate percentages.

Recommendation: Use the revised version of the question asking for percentages.

Question 13

5. Were any of the newly hired employees in April 2022 for positions open for 30 DAYS OR LESS?

• Yes

• No -> GO TO Q16

Finding: The question on whether the newly hired employees were for positions open for 30 days or less was generally well understood and clear to respondents. They typically knew this information off the top of their heads, or consulted company records or other staff to confirm their answers.

Recommendation: No question changes are recommended.

Question 14

5. Thinking about positions that took 30 DAYS OR LESS to fill, how many of these required a professional, state, or industry license or certificate?

• All

• Some

• None

• Don’t know

(Revised version) Thinking about positions that took 30 DAYS OR LESS to fill, about what percent of these required a professional, state, or industry license or certificate?

Enter approximate percent (E.g., 10 = 10%)

[Numeric text box]

Finding: The question on positions that 30 days or less to fill and how many required a license or certificate was well understood and no problems were observed. Several respondents remarked that these types of licenses or certificates are preferred, but not a requirement. As noted above in the General Findings section, a revised version of this question was later tested that asked for percentages instead of Likert scale options, as the Likert scale was not yielding meaningful distinctions. These responses to the updated online questionnaire revealed no difficulty in respondents being able to provide approximate percentages.

Recommendation: Use the revised version of the question asking for percentages.

Question 15

5. For open positions that took 30 DAYS OR LESS to fill, how many of these required a bachelor’s degree (4-year degree, BA or BS) or higher?

• All

• Some

• None

• Don’t know

(Revised version) For open positions that took 30 DAYS OR LESS to fill, about what percent of these required a bachelor’s degree (4-year degree, BA or BS) or higher?

Enter approximate percent (E.g., 10 = 10%)

[Numeric text box]

Finding: The question on positions that took 30 days or less to fill and how many required a BA degree was well understood and no problems were observed. Several respondents remarked that a BA degree is preferred, but not a requirement. As noted above in the General Findings section, a revised version of this question was later tested that asked for percentages instead of Likert scale options, as the Likert scale was not yielding meaningful distinctions. These responses to the updated online questionnaire revealed no difficulty in respondents being able to provide approximate percentages.

Recommendation: Use the revised version of the question asking for percentages.

Question 16

This next section asks about current job vacancies that we refer to as “open positions.” This includes any paid position, new or unoccupied, that this business location is taking active steps to recruit or hire for to fill the position. For positions (occupations) with multiple vacancies, report the number of candidates you would be willing to hire for that position.

5. Does this location currently have any open positions that you are looking to fill?

Exclude positions that you are not currently advertising or seeking to fill.

• Yes

• No -> GO TO Q20

Finding: Respondents were asked for feedback on the transition text to the section on vacancies. Most respondents seemed to understand the language and understood what “open positions” referred to and found the text helpful. The question on whether the location has any open positions was well understood and clear to respondents. Most respondents knew they usually have openings and verified this by looking at company records or asking another co-worker to confirm. One respondent mentioned he visited the company’s external website to see if any open positions were currently posted.

Recommendation: No question changes are recommended.

Question 17

5. How many open positions does this location currently have?

[Numeric text box]

Finding: The question on how many open positions the location currently has was well understood and easy for respondents to answer. Most indicated these are positions that are currently being recruited and advertised for.

Recommendation: No question changes are recommended.

Question 18

5. How many of these open positions are eligible for full time telework (remote work)?

[Numeric text box]

Finding: The question on how many open positions are eligible for full-time telework was clear and well understood by respondents. It was easy for most respondents to answer. A few respondents noted they do not track this information in any official way in company records, so they had to infer the answer based on the job titles of the open positions or the team or location they work for.

Recommendation: No question changes are recommended.

Question 19

5. How many of these open positions have been open for MORE THAN 30 days without being filled?

[Numeric text box]

Finding: The question on how many of the positions have been open for more than 30 days without being filled was generally well understood for most respondents. Several respondents noted that due to the large amount of turnover, this question was a little complicated to answer (similar to the feedback from Q9/10). For example, one respondent noted that he wasn’t exactly sure how to answer the question due to the large amount of turnover at the company, so it would depend on how “open positions” are counted – by the position or the person. He stated one position was filled and another was turned over within the same month, so there is a “net one opening” by his count, and felt it would be easier for himself and other large firms to tie it down to individuals. Despite some minor difficulty due to turnover for these positions, all respondents were able to understand the question and provide an answer.

Recommendation: No question changes are recommended.

Question 20

5. Has this location had any open positions in the past year?

• Yes

• No -> GO TO Q23

Finding: The question on whether the location had any open positions in the last year was generally easy for respondents to understand, and most thought about the previous 12 months as a reference period. However, one respondent said he thought of “only the current year, 2022” and gave his answer in terms of open positions since January 2022. In addition, several respondents noted that this question was redundant with the prior question on whether they did any hiring in April of 2022 (Q5), and suggested that only respondents who answered “no” to Q5 should question should receive Q20.

Recommendation: Change the question wording to “Has this location had any open positions in the last 12 months?”

Question 21

5. For positions that require a bachelor’s degree (4-year degree, BA or BS) or higher, what are all the ways that this location advertised for any open positions in the last year?

Select all that apply.

• Online job boards/hiring platforms

• Company website

• Advertisements in periodicals (newspapers, magazines, trade publications: electronic or paper)

• Recruiting firm

• Word of mouth/social media

• Physical advertisement (sign in store, billboard, etc.)

• Not applicable

• Other

Question 22

5. For positions that do NOT require bachelor’s degree (4-year degree, BA or BS) or higher, what are all the ways that this location advertised for open positions in the last year?

Select all that apply.

• Online job boards/hiring platforms

• Company website

• Advertisements in periodicals (newspapers, magazines, trade publications: electronic or paper)

• Recruiting firm

• Word of mouth/social media

• Physical advertisement (sign in store, billboard, etc.)

• Not Applicable

• Other

Finding: The questions asking respondents to report ways that they advertised positions (both for those requiring a BA degree or higher and those not requiring it) was generally easy for respondents to understand and the difference between the two questions was obvious. When asked if they would add items to the list of ways to advertise open positions, a few respondents mentioned contacting temporary staffing agencies might be also be included as they had done so in the past. A couple of respondents also suggested separating out Word of mouth and Social media from response option number 5; some respondents only did one or the other and thought of them differently.

Recommendation: Change the question wording to mirror question 20, “…in the last 12 months.” No response option changes are recommended. Temporary help agency employees are not employees of the surveyed establishment. Hiring a temp would be in place of hiring at the establishment. Social media and word-of-mouth are generally considered the same and will be kept as one response option.

Summary of Questionnaire Changes

1. Telework Questions – No changes

We reviewed the telework questions text and decided to leave the questions in their original format. As noted, the telework description includes “work from home” and telework as a concept is pretty well known and understood to be “paid work,” as in employees are paid to work from an alternate location. It’s possible that by including ‘work from home’ in the body of the question we’d be deviating from the paid work concept and would unnecessarily open up the telework questions to be answered affirmatively, for every employee that might work 9-5 in the office and occasionally check their email from home.

2. Question 6 - How many new hires in April 2022?

We looked at a ratio of average monthly employment to the number of new hires reported in April. Most respondents reported values that were in line with expectation. About 1% of respondents reported values that were higher than expected. One reason for this could be that they are providing a firm-level responses. We are adding text to the section introduction and question 6 reiterating “this location only, not including other locations of this company.”

We have concluded that asking for hires data for more than one month would be overly burdensome for large establishments having to review records to obtain the data and could also impact the smaller establishments relying on recall of the prior month to answer the questions about hires. The reference for hires will be July 2022.

To clear up any confusion concerning hires that that may leave a position shortly after being hired, we are updating the text to “Include employees that were hired, even if they have not yet formally started, or have left the position since being hired.

3. Question 8 – What did businesses do to attract more applicants

Added “Expanded telework or remote work” to response options.

4. Questions 11/12, 14/15 - Licensing and degree requirements

These questions ask about professional license or bachelor’s degree requirements for new hires that took both >30 and <30 days to hire. The original question gave respondents 3 options to select: All, Some or None. Initial results showed that most respondents with >5 hires are selecting “Some” for both >30 and <30 days, yielding results that will not be meaningful in making comparisons or distinctions.

These questions were updated to ask for approximate percentages, rather than the 3-level Likert scale options. The follow up email blast tested respondent ability to provide approximate percentages. These responses to updated online questionnaire revealed no difficulty in respondents being able to provide the approximate percentages.

5. Question 20 – Hires in the last year

We will change the question to hires in the last 12 months.

Questions 21 and 22 –

We will change the questions to “…in the last 12 months.”

No changes to the response options listed. Temporary help agency employees are not employees of the surveyed establishment. Hiring a temp would be in place of hiring at the establishment. Social media and word-of-mouth are generally considered the same.

7. Option for more feedback

During cognitive interviews, many respondents reported that they wanted to provide additional information about the difficulties that they are having with hiring. We also received a detailed email back from a respondent with additional information about difficulties hiring. For this reason, we are adding a line to the Thank-You page with a link for respondents to be able to provide additional feedback, if they so desire.

Revised Questionnaire

The Bureau of Labor Statistics (BLS) is looking to understand telework, hiring, and vacancies at your business. This information is particularly critical in understanding how the labor market has changed because of the COVID-19 pandemic. You’ll see questions about telework at your location both currently and before the pandemic, and questions about newly hired workers and existing vacancies at your business location.

Please continue to this short survey about this business location’s telework policies, recent experience in hiring and current vacancies.

Please answer the following questions for this business location only.

Telework is a work arrangement that allows an employee to work at home, or from another remote location, by using the internet or a computer linked to one’s place of employment, as well as digital communications such as email and phone.

1. Do any employees at this location CURRENTLY telework in any amount?

• Yes

• No --> Skip to question 3

2. In a typical week, what percent of employees CURRENTLY telework in the following amounts?

Answers should total 100%

[____] All the time (remote employee)

[____] Some of the time (some work hours or days via telework)

[____] Rarely or never (rare occasions of telework, or full-time on-site)

3. In the next 6 months, does this location expect the amount of time that employees are permitted to telework to…

• increase.

• decrease.

• stay the same.

4. In February 2020, before the coronavirus pandemic began, did any employees at this location telework in any amount?

• Yes

• No

• Don’t know

• Location not in business in February 2020

This next section refers to any hiring done by this location in July 2022. When answering, please include only new hires at this specific location, and not any other locations of the company.

5. In July 2022, did this location hire any new employees?

Include employees that were hired, even if they have not yet formally started or have left the position since being hired.

• Yes

• No --> Skip to question 16

5. In July 2022, how many new employees did this location hire? Please include only new hires at this specific location, and not any other location of this company. [____]

5. How many of these new employees hired in July 2022, will telework all the time (be remote employees)? [____]

5. For positions filled in July 2022, did this location do any of the following to attract more applicants?

Select all that apply.

• Expanded advertising

• Started using recruiters/talent agencies

• Increased starting pay

• Offered hiring bonuses

• Offered more hours (e.g., changed position from part-time to full-time)

• Reduced qualifications (e.g., education or experience)

• Expanded benefits

• Expanded telework or remote work

• None of the above

5. Were any of the newly hired employees in July 2022, for positions open for MORE THAN 30 days?

• Yes

• No --> Skip to question 14

5. How many of the newly hired employees in July 2022, were for positions open for MORE THAN 30 days? [____]

5. For positions that took MORE THAN 30 days to fill, about what percent of these required a professional, state, or industry license or certificate?

Enter approximate percent (e.g., 10 = 10%) [____]

5. For positions that took MORE THAN 30 days to fill, about what percent of these required a bachelor’s degree (4-year degree, BA or BS) or higher?

Enter approximate percent (e.g., 10 = 10%) [____]

5. Were any of the newly hired employees in July 2022, for positions open for 30 DAYS OR LESS?

• Yes

• No --> Skip to question 16

5. Thinking about positions that took 30 DAYS OR LESS to fill, about what percent of these required a professional, state, or industry license or certificate?

Enter approximate percent (e.g., 10 = 10%) [____]

5. For open positions that took 30 DAYS OR LESS to fill, about what percent of these required a bachelor’s degree (4-year degree, BA or BS) or higher?

Enter approximate percent (e.g., 10 = 10%) [____]

This next section asks about current job vacancies that we refer to as “open positions.” This includes any paid position, new or unoccupied, that this business location is taking active steps to recruit or hire for to fill the position. For positions (occupations) with multiple vacancies, report the number of candidates you would be willing to hire for that position.

5. Does this location currently have any open positions that you are looking to fill?

Exclude positions that you are not currently advertising or seeking to fill.

• Yes

• No --> Skip to question 20

5. How many open positions does this location currently have? [____]

5. How many of these open positions are eligible for full time telework (remote work)? [____]

5. How many of these open positions have been open for MORE THAN 30 days without being filled? [____]

5. Has this location had any open positions in the last 12 months?

• Yes

• No --> Skip to Submit

5. For positions that require a bachelor’s degree (4-year degree, BA or BS) or higher, what are all the ways that this location advertised for any open positions in the last 12 months?

Select all that apply.

• Online job boards/hiring platforms

• Company website

• Advertisements in periodicals (e.g., newspapers, magazines, trade publications: electronic or paper)

• Recruiting firm

• Word of mouth/social media

• Physical advertisement (e.g., sign in store, billboard, etc.)

• Not applicable

5. For positions that do NOT require bachelor’s degree (4-year degree, BA or BS) or higher, what are all the ways that this location advertised for open positions in the last 12 months?

Select all that apply.

• Online job boards/hiring platforms

• Company website

• Advertisements in periodicals (newspapers, magazines, trade publications: electronic or paper)

• Recruiting firm

• Word of mouth/social media

• Physical advertisement (sign in store, billboard, etc.)

• Not Applicable

Time of completion is estimated to about 5 minutes. This estimate includes time for reviewing instructions, searching existing data sources, gathering and maintaining data needed, and completing and reviewing this information. If you have any comments regarding these estimates or any other aspect of the survey, please contact [email protected]. The OMB control number for the survey is 1220-0198 and it expires on 07/31/2024. Without a currently valid OMB number, BLS would not be able to conduct this survey.

16