Attachment K Work Schedules Supplement Cognitive Testing Report

Attachment K

TO:

Karen Kosanovich
Office of Employment and Unemployment Statistics

FROM:

Robin Kaplan and Victoria Narine
Office of Survey Methods Research (OSMR)

DATE:

November 27, 2023

SUBJECT: Report on Cognitive Testing for the 2024 Current Population Survey Work
Schedule Supplement
Introduction
The Bureau of Labor Statistics has periodically collected information on work at home and
workers with flexible and shift schedules in work schedules supplements (WSS) to the Current
Population Survey (CPS) over several decades. The most recent work schedules supplement was
conducted in 2004. The supplement collected information on work schedules, work at home, and
details for respondents’ main job and second jobs. The CPS will be conducting a work schedules
supplement again in 2024 with modified questions and updated concepts. These include:
1.
2.
3.
4.
5.
6.
7.

Beginning and ending hours of work
Days of the week worked (how many and which)
Flexible schedules, including employer-offered flexitime program
Shift work
Reason for non-daytime shift
Work at home arrangements, include hours, frequency, and reasons for working at home
Second jobs, including reason for working more than one job

Because the last WSS was conducted in 2004, updated cognitive testing of the supplement
questions and concepts may need revision and modernization. BLS recently presented the WSS
concepts to both its Data User Advisory Committee and Technical Advisory Committees for
ideas on concepts that would be relevant to a 2024 WSS. Both groups emphasized the
importance of collecting information on flexible work schedules and work at home, as these have
rapidly evolved in the past three years. Additional topics, such as how far in advance workers
know their schedule and how much input a worker has in determining their schedule were also
identified as important. Distinctions between work at home, telework, and remote work were
recognized as important constructs to potentially assess in the supplement, however,
respondents’ understanding of these terms may be inconsistent. Thus, updating the WSS and
conducting additional cognitive testing to assess respondents' understanding of these concepts is
crucial prior to fielding the supplement.
The purpose of the cognitive testing for the CPS Work Schedule Supplement was to ensure that
(a) the questions show construct validity and will serve to address the issues outlined above, (b)
respondents to the survey understand the terminology used in the questions, (c) respondents have
the requisite knowledge to answer the questions for both themselves and for other members of
their household, and (d) whether respondents have difficulty or sensitivity answering the
supplement questions.
1

Methods
OSMR conducted two rounds of cognitive testing, for a total of 19 participants (13 in Round 1
and 6 in Round 2). Using an iterative design, questions were revised between the first and second
rounds of testing. See Appendix A, B, and C for the questionnaires used in the first and second
round of testing and in the online study.
All participants indicated that they worked full or part time for the government, a private
company, or a non-profit. Participants were recruited via advertisements on Craigslist and were
offered a $50 incentive. OSMR interviewers administered a brief set of CPS labor force
questions to confirm that they were currently working, as well as a household roster of other
adults aged 15 and above. Afterward, participants answered the WSS questionnaire for
themselves and up to two additional household members. After completing the questionnaire,
participants answered debriefing questions about their experience answering the questions.
Online testing from a nonprobability panel was also conducted. A total of 1000 participants were
screened to determine eligibility, and 500 were invited to complete the web-probing study.
Participants were asked the same questions using similar probes to the cognitive testing study,
but in a self-administered format. Results from the online testing are also included to help
support the recommendations.
Participants
Round 1
A total of 13 participants were included in the first round of testing. The average age of the
participants was 42 years old, with a range of 30 to 56 years old. All participants had a
Bachelor’s degree or higher. A total of 6 participants reported being employed full-time, 1 was
employed part-time, and 6 were self-employed. Participants represented a mix of different
occupations and whether they were self-employed versus worked full-time or part-time. In
addition to the 13 self-reports by the participants, 10 proxy reports were also provided.
Round 2
A total of 6 participants were included in the second round of cognitive testing. The average age
of the participants was 42 years old, with a range of 27 to 52 years old. One participant had some
college and six had a Bachelor’s degree or higher. A total of 2 participants reported being
employed full-time, 2 were employed part-time, and 2 were self-employed. Participants
represented a mix of different occupations. In addition to the 6 self-reports by the participants, 6
proxy reports were also provided.

2

Web-Probing Study
A total of 491 participants completed the web-probing survey. The average age was 39 years old,
with a range from 18 to 85 years old. All participants (49% female) reported working full-time
(78%) or part-time (13%) or were self-employed/business owners (9%). Participants represented
a mix of different occupations, and the median education level was Bachelor’s degree or higher.
Results
NEWFlexDay
Cognitive interviews
Round 1

Do you have a flexible schedule that allows you to vary or make changes in the days you
work?
1. Yes
2. No
This question was generally well understood, and no major problems were observed in proxy
response. However, the question sometimes seemed odd for salaried workers who always work
Monday to Friday:
“I don't know very many people who have flexible schedules where they can change the
days, but there are many industries out there.”
These participants sometimes also included the HOURS they usually work in their response to
this question. They had a hard time separating the days they work from the times when thinking
about their schedules:
“I do have flexible schedules in terms of the hours I work but not necessarily the days.”
The terms “flexible schedule” or “work schedule” were also inconsistently understood. Some
participants thought of the days and times they work, or just the times worked, or the meetings
they have on the calendar for the day:
“Because a lot of what I do is event or deadline driven, i can reprioritize what I have to
do and that changes the schedule. For example, I can work with clients on a daily
schedule to deal with as opposed to a schedule that starts later in the day.”

3

“I should have set schedule, but sometime meetings get scheduled outside my hours. I
don't usually have flexibility, but sometimes need to be flexible. If I take meetings during
nonwork hours I might take off early on Friday.”
Other participants answered in terms of being able to vary the days if they really wanted to, but
they never actually do this in practice because their job has traditional, Mon-Fri, 9-5 hours.
Others mentioned they could do this on occasion, but it’s not something they would do regularly:
“From a professional perspective, he cannot change the days he works. He could on
occasion, but it’s not something he would do regularly.”
For Round 2, we recommended capitalizing the word “DAYS” to help respondents understand
the question asked about varying the days (not hours) worked. In addition, we recommended
removing the term “flexible schedule” from the question stem to avoid confusion and
inconsistent comprehension of this concept.
Round 2

The question was modified in Round 2 to capitalize the word “days,” the term “flexible
schedule” was removed from the question stem, and new instructions were written to exclude
vacation or sick days from their response. Additionally, a read if necessary instruction was added
to clarify the question is asking about whether respondents can choose which days of the week
they work.
Are you allowed to vary or make changes in the DAYS you work? Exclude vacation or
sick days. Read if necessary: For example, choosing which days of the week you work.
1. Yes
2. No
During debriefing, participants were also asked to answer the following version of the same
question:
[Does your employer/Does your job] allow you to change which days of the week you
work? Exclude vacation or sick days.
1. Yes
2. No
During debriefing, participants seemed to understand both questions and gave the same answers
to each of the questions. No major issues occurred in proxy response. Some participants
expressed confusion about the intention of the question:
“I’m confused about whether you mean routinely or just on occasion. I can take off a day
here and there if I need to, but I can't just work Tuesday through Saturday.”

4

For others, there was a fine line between answering yes or no. These participants mentioned that
they are allowed to make these changes if there’s an important reason to do so, but it’s expected
they will work certain days of the week. Some didn’t know if this was a formal policy, but that
they’ve always been expected to work only Mon-Fri.
The instructions to “Exclude vacation or sick days” were not always well-understood. One
participant with low English ability did not understand this phrase. A couple participants
expected to see “personal days” in the list of items to exclude in addition to vacation or sick days
or included taking personal days in their response and then making up the time during the week
or another day. Some participants seemed to include sick days in their response:
“We both have work situations where if we needed to take a day off for whatever, we
would be able to just go ahead and do that and structure the rest of our work week around
that. For childcare, travel, or medical appointments.”
Some participants noted that while they CAN make changes to the days they work, they almost
never do this in practice. This made answering the question a little more difficult, as it became
hypothetical. They would usually answer yes, with a caveat that they don’t often do so.
Participants remarked the question might resonate better with shift workers or non-salaried
workers, who can cover shifts for one another.
There were mixed findings on preference between the two versions of the question, with a slight
preference for the question asked during the debriefing. However, the self-employed or those
with extremely flexible schedules preferred the original question, while those with less flexibility
preferred the question asked during debriefing. It was noted that the phrasing of “vary or make
changes” encompasses more of the flexibility their jobs afford. Others commented that the
question asked during debriefing seemed more direct and concrete, making it easier to
understand the intention. Most people noted that they thought of their manager or company as
their “employer,” and the question implied that they would need to ask permission from their
employer to change the days they work. The extra instruction sometimes caused participants to
forget the original question stem, and the interviewer had to repeat the question.
Recommendations

We recommended moving the question to directly after the question about which days of the
week people USUALLY work to improve questionnaire flow. Respondents will have just
answered about which days of the week they work, and then they can be asked if they can vary
those days. This might increase question comprehension. An alternative would be switching the
order so that the HOURS question comes first to aid comprehension.
We also recommended removing the instructions to “exclude vacation or sick days,” as not all
workers have this type of leave plan. Some respondents may not get paid time off at all, or they
may have a PTO plan. This could instead be an interviewer instruction to read if necessary.

5

Web-probing study

Before answering this question, web-probing participants answered questions on the number of
days per week they work and which days of the week they work. Most participants in the webprobing study indicated that they work 5 days per week (72.3%), with most days worked being
Monday-Friday. A total of 15.3% of participants reported working on Sundays and 19.8%
reported working on Saturdays.
This question was modified as follows for the web-probing study.
Does your [employer/job] allow you to change the DAYS of the week you work?
Please respond yes if your employer allows you to change your days, even if you haven’t done
this. Exclude taking time off for vacation, sick days, or other time off.
Distributions to this question by type of worker (wage and salary workers, that is, those who
work full-time or part-time, vs. self-employed/business owners) are displayed below.

Full-time (n=384)
Part-time (n=65)
Self-employed/business owners (n=42)

Does your [employer/job] allow you to change
the DAYS of the week you work?
Yes (n=229)
No (n=262)
37.2%
62.7%
70.7%
29.2%
95.2%
4.8%

Most part-time workers (over 70%) indicated that their employer allows them to change the days
they work, as opposed to only 37% of full-time workers. As expected, nearly all (over 95%) of
self-employed participants indicated that their job allows them to change the days of the week
they work. The association between type of employment and being allowed or able to change the
days worked was significant, χ2(2)= 68.7, p < .001.
A follow-up probe was administered to participants who indicated “yes” to the previous
question:
On your job, have you ever changed the DAYS of the week you worked? (Yes/no)
Of the 229 participants administered the question, 79% of participants answered that they had
changed the days of the week they worked, while 21% answered no. Crosstabs between type of
employment (full-time, part-time, or self-employed) and whether participants had changed the
days they worked in the past are displayed below.

Full-time (n=143)

On your job, have you ever changed the DAYS of the week
you worked?
Yes (n=181)
No (n=48)
73.4%
26.6%
6

Part-time (n=46)
Self-employed (n=40)

86.9%
90.0%

13.1%
4.0%

As expected, a high proportion of self-employed participants (90%) reported they had changed
the days they work. While many part-time participants (87%) also reported changing the days
they worked, a large share of participants who work full-time (over 70%) also reported having
done so in the past. The association between type of employment and having changed the days
worked was significant, χ2 (2) = 7.4, p = .03.
Crosstabs between NEWFlexDay and a later question on whether participants work a daytime
schedule or some other schedule, were also conducted to determine whether there was an
association between changing the days worked and type of schedule worked (daytime versus
some other schedule).
On your job, do you USUALLY work a daytime schedule
or some other schedule?
Does your employer/job
allow you to change the
DAYS of the week you
work?
Yes (n=229)
No (n=262)

Daytime
(n=446)

Some other schedule
(n=45)

42.8%
57.2%

84.4%
15.6%

A total of 84.4% of participants who worked a non-daytime schedule reported that they are able
to change the days worked, as opposed to only 42.8% of participants who worked a daytime
schedule. This suggests that those working shifts may more easily be able to change the days
worked. The association between being able to change the days of the week worked and working
a daytime schedule or some other schedule was significant, χ2 (1) = 28.5, p < .001.

Final Recommendations

Evidence from the cognitive interviews and web-probing studies suggest participants may not
have understood the intent of the question on changing the days they work. Cognitive interview
participants did not always know the answer or included use of personal days in their answer.
Web-probing participants may have overreported on the ability to change the days they work,
given most worked Mon-Fri full-time schedules, suggesting potential comprehension issues. The
question might work better for those who indicate they do shift work. We recommend either
dropping the question or restricting it to those who work a non-daytime schedule and/or work
part-time.
SMJFlex
7

Cognitive interviews
Round 1
(Do you/Does NAME) have flexible work hours that allow (you/him/her) to vary or make
changes in the time (you/he/she) (begin/begins) and (end/ends) work?
1. Yes
2. No
This question was generally well understood, and no major problems were observed in proxy
response. Most participants easily answered the question without needing clarification. A few
participants expressed some difficulty, however:
One participant – a legal secretary working in a “pretty traditional” law firm -- answered
“Hmmmm. Sort of.” When probed for a “yes” or “no” response, the participant followedup with “Technically, no. But they’re very flexible.” This participant ultimately settled on
“no” as a response.
Another participant interpreted “flexible work hours” as a function of pay or income.
This participant worked as a food delivery driver and stated that certain times during the
day were more lucrative than others, leading them to rely on making deliveries during
these times to meet her family’s financial needs. Thus, the participant perceived their
work hours as being inflexible because of the necessity to work during those busy times,
explaining “If I were to start work later than I usually do, I would be missing hours. Then
I would have to stay working later into the morning when there aren’t so many deliveries
available. I would miss hours in the beginning, and I wouldn’t be able to make up for it in
the morning…I will lose income…”
The third participant perceived their work schedule as similarly inflexible, albeit because
of the need to accommodate multiple clients. As a language therapist, this participant
worked a split schedule (i.e., hours before and after school) and had only a few hours
during each period when they could meet with clients. They said that “Each semester of
the school year, I proved a schedule for each child. Once set, it’s a full schedule and I
can’t change it. I can only see students before or after school hours.”
Interestingly, two of the three participants who selected “no” to this question identified as being
self-employed and a contract worker, indicating that the concept of flexibility may be selfdefined. In fact, the two self-employed participants both defined flexible work hours as the
ability to choose when they work, an option that they didn’t feel applied to them.
The remaining participants interpreted this question as asking about whether they were able to
shift their schedule to accommodate personal or family matters. Self-employed participants
tended to answer “yes” to this question because they can control when they meet with clients.
Full-time wage and salary workers who answered “yes” to this question did so because they were
8

able to shift their schedule depending on important work or personal commitments. In these
cases, participants commented that it didn’t necessarily matter when they worked, as long as they
got the job done. For example, one participant said, “My company is pretty relaxed on when you
work your hours as long as the work gets completed…As long as I get my 8 hours in, 40 hours
per week, there are no issues.”
Similarly, some participants thought that flexible work hours could mean working 30 hours one
week and 50 hours the next, depending on the workload. While these participants said they did
have a flexible schedule, some said that just because they can work flexible hours didn’t mean
that they do it in practice because it looks unprofessional. When talking about their spouse, one
participant said, “…he has the ability to do it, although it may not be a best practice to make
these changes.” There was a tension between being allowed to shift work hours versus the actual
behavior.
Round 2
Are you allowed to vary or make changes in the TIME you start or end work?
1. Yes
2. No
During debriefing, participants were also asked to answer the following version of the same
question:
[Does your employer/Does your job] allow you to change the time you start and end work?
1. Yes
2. No
In Round 2, the question was modified to account for the tension between being allowed to work
flexible work hours and the actual behavior. Participants were asked about the second question
during the debriefing. Both questions were generally well understood and there were no major
issues with proxy response. Like the participants in Round 1, several participants stated that
when they worked didn’t matter if tasks were completed by the deadline. For example:
An attorney stated, “My employer has core hours, but no one is monitoring. We’re
evaluated at the end of the year on billing hour requirements, but they’re not paying
attention otherwise.” When probed on what the phrase “are allowed to” meant, there was
a range in who was doing the allowing.
One participant thought the phrase meant “could you independently change hours”
whereas another participant thought it meant “does your employer allow the flexibility to
alter start/end times as needed.” That same participant said their spouse’s work colleagues
would be the ones doing the allowing.
All participants gave consistent answers to both questions but had a slight preference for second
question asked during debriefing because they felt like it was more direct. One participant said,
9

“It took out the ambiguity of what it means to be allowed. The one on the left implies ‘You could
do it, but someone could complain.’” Participants also mentioned formal policies being tied to
the employer, making the question clearer. However, self-employed participants preferred the left
version specifically because of the word “vary,” which they felt represented the flexibility
inherent in being self-employed.
Recommendations

It was recommended that “time” is capitalized to emphasize the difference between time and
days. Other possible wording options included:
• Does your job allow you to change or vary the TIME you start or end work?
• Are you able to change the TIME you start or end work?
• At your (main) job, can you make changes to the TIME you start or end work?
Web-probing

For the web-probing study, the question was modified as follows: “Does your employer/job
allow you to change the TIME you start and end work?” Response by employment type is
displayed below.
Does your employer/job allow you to change the
TIME you start and end work? (N=490)
Yes (n=275)
No (n=215)
Employed full-time (n=383)

51.7%

48.3%

Employed part-time (n=65)

55.4%

44.6%

Self-employed (n=42)

97.6%

2.4%

As expected, almost 100% of self-employed participants answered yes to this question.
Responses for those who work full-time or part-time were somewhat split, with most indicating
they could change their start and end times.
Web-probing participants (excluding the self-employed) were administered a probe asking them
who or what best describes what they thought of as their “employer” when answering the
question. Participants were evenly split between considering their supervisor to be their
employer or their organization as a whole; the other categories were sparsely chosen.

My immediate supervisor or boss

Which of the following best describes who or
what you considered to be “your employer”?
(n=448)
44.0%

My specific work group

4.7%
10

My organization as a whole

44.4%

A specific or formal workplace policy

2.5%

The needs of clients/costumers/colleagues

2.7%

I decide/my personal needs

1.1%

Other

0.7%

Final recommendations

The revised version of the question used in the web-probing study seemed to work well. Most
participants considered their employer to be their immediate supervisor or their organization as a
whole and were evenly split between these two categories at 44% each. Differences in
interpretation may introduce slight measurement error. If respondents should think of their
employer as a supervisor or organization, a definition or instructions could be included to read if
necessary. The question used for the self-employed also seemed to work well. However, the
revised version of the question put more emphasis on the employer controlling the schedule,
which did not resonate as well for those with extremely flexible work hours but are not selfemployed.
We recommend either using the revised version of the question, “Does your employer/job allow
you to change the TIME you start and end work?” (with a possible read if necessary instruction
to define who respondents should consider as their employer – either their supervisor or
organization); or consider the version of the question used in the 2024 ATUS Leave Module
could be considered as a possible alternative: “Do you have flexible work hours that allow you to
vary or make changes in the times you begin and end work?” which places less emphasis on the
employer or organization controlling the schedule.
SMJDaytime/SMJShift
On your job, do you USUALLY work a daytime schedule or some other schedule?
1. A daytime schedule – most work occurs between 6 am and 6 pm
2. Some other schedule
Which of the following best describes the hours you USUALLY work at this main job:
an evening shift, a night shift, a rotating shift, a split shift, an irregular schedule, or some
other schedule?
1. An EVENING shift – most work occurs between 2pm and midnight
2. A NIGHT shift – most work occurs between 9pm to 8am
3. A ROTATING shift – hours change periodically from days to evenings or night
4. A SPLIT shift – with two distinct periods each day
5. An irregular schedule
6. Some other shift

11

Cognitive interviews

The same version of this question was asked in both rounds of cognitive interviews. Most
participants answered this question without any issues for themselves for other household
members. Some participants needed the probe read specifying whether most work occurs
between 6am and 6pm to correctly answer the question. For example, a few participants wanted
to select ‘Some other schedule’ because they work with clients in other time zones and answered
that their schedule was irregular, but later noted that most of their work occurs during the day.
Another participant rotated working days and evenings and selected an irregular schedule. Most
participants did not report doing shift work, but said these categories made sense to them. One
participant who worked a night shift was easily able to select a category. Due to limited testing,
we recommended additional testing in the web-probing survey.
Web-probing study

This series of questions was modified as follows for the web-probing study. Participants were
asked an additional question to determine if daytime workers consider themselves “shift
workers,” along with an open-ended probe for participants to describe the type of shift they
On your job, do you USUALLY work a daytime schedule or some other schedule?
1. A daytime schedule – most work occurs between 6 am and 6 pm
2. Some other schedule
Even though you usually work a daytime schedule, are you a shift worker?
1. Yes
2. No
(If yes), Please describe the shift you work. [open-end]
Which of the following best describes the hours you USUALLY work at this main job:
an evening shift, a night shift, a rotating shift, a split shift, an irregular schedule, or some
other schedule?
1. An EVENING shift – most work occurs between 2pm and midnight
2. A NIGHT shift – most work occurs between 9pm to 8am
3. A ROTATING shift – hours change periodically from days to evenings or night
4. A SPLIT shift – with two distinct periods each day
5. An irregular schedule
6. Some other shift
Does your [employer/job] allow you to change the shift you work?
1. Yes
2. No
(if yes) How frequently do you change the shift you work?
1. Frequently
2. Occasionally
3. Rarely
12

work. Those who work other shifts were asked to select the type of shift they work, then
answered a follow-up question on whether they change their shift and how often they change
their shift.
Most participants reported working a daytime schedule (90.8%). The table below shows the
response distributions to these questions by type of employment.

Full-time (n=384)
Part-time (n=65)
Self-employed (n=42)

On your job do you USUALLY work a daytime schedule
or some other schedule? (N=491)
Daytime (n=446)
Some other schedule (n=45)
92.4%
7.6%
86.1%
13.9%
83.3%
16.7%

Participants who worked a daytime schedule were asked a follow-up question about whether
they are shift workers. Most participants answered “no” (72.5%), with over one-fourth answering
“yes” (27.5%). The table below displays responses to this follow-up question by wage and salary
workers who work full-time versus part-time.

Full-time (n=355)
Part-time (n=56)

Even though you usually work a daytime schedule, are you
a shift worker? (n=411)
Yes (n=113)
No (n=298)
25.6%
74.4%
39.3%
60.7%

Participants who responded “yes” were asked to describe the type of shift they worked. Most
participants seemed to misinterpret the intended meaning of “shift work” and described the hours
they typically work, which seemed to be regular business hours. It’s unclear if these participants
considered daytime to business hours to be a “shift,” or if their employer refers to these hours as
“shifts.”
•
•
•
•
•
•
•
•

“Day shift; 8AM to 4PM.”
“Typically a morning shift around 8am till 6 pm. But there are times I will have to stay
late.”
“My shift is from 7:00am-4:00pm.”
“I usually work 9am to 5pm. So standard daytime shift.”
“I work a morning shift that is 8 hours, typically from 8am - 4pm but sometimes from
9am - 5pm.”
“Our shifts are between the hours of 6:30 am - 6:30 pm. I usually work the 7 am -4 pm
shift.”
“I work a 9-5 shift everyday.”
“I work from 8:30 - 5:30 Monday – Friday.”
13

•

“As an accountant, I typically work a standard shift from 9 a.m. to 5 p.m., Monday
through Friday except Wednesday. This aligns with the regular business hours and allows
me to engage with clients, attend meetings, and collaborate with colleagues during the
standard workweek.”

A couple participants expressed uncertainty about indicating that they work in shifts because
their schedule doesn’t change very much, although it may still be considered a “shift”:
•
•

“I typically work from 10 am-6 pm, it can sometimes vary, but typically that is the type
of shift that I work.”
“I'm a ER nurse so technically I'm a shift worker but it's pretty standard 8am - 5pm with
breaks. Occasionally during extreme short staff times I work nights.”

Other participants seemed to understand the intent of the question and explained the types of
shifts they work:
•
•
•
•
•
•

“A mixture of opening, closing and mid-shift.”
“My type of shift work is a rotating shift. I work 6am to 12pm one day and 12pm to 6pm
the next day.”
“I work three shifts as the Senior program manager. I sometimes work 6am-2pm, 2pm10pm, or 10pm-6am.”
“I work at a hospital, so I usually work in the morning but I can work a mid-shift or
night-shift if I want to as well.”
“I do mostly day time hours, but sometimes there are night shifts. It all depends.”
“I alternate between all three shifts. 1st, 2nd and 3rd.”

Participants who indicated they did not work a daytime schedule (n=45) were asked to select
which of the following options best describes the hours they work. Results are displayed below.

Evening shift

Which of the following best describes the hours you USUALLY
work at this job? (n=45)
26.7%

Night shift

15.6%

Rotating shift

22.2%

Split shift

6.7%

An irregular schedule

26.7%

Other

2.2%

A total of 38 participants who worked full- or part-time (not self-employed) answered the followup question that asked Does your employer/job allow you to change the shift you work? Of those
participants, 73.7% indicated yes, they can change the shift they work. A total of 7 self-employed

14

participants answered the question, and as expected, all of them indicated that their job allows
them to change the shift they work.
A final probe was asked about how often participants change their shift. Most indicated that they
did so frequently or occasionally.
How often do you change the shift you work? (n=35)
Frequently

22.9%

Occasionally

45.7%

Rarely

31.4%

Final Recommendations

Overall, participants understood the question on whether they work a daytime or other schedule.
However, the web-probing participants seemed to overreport on being daytime shift workers,
considering their Mon-Fri, 9 to 5 schedule to be a “shift.” Thus, we recommend dropping this
follow-up question. Depending on data user needs and time constraints, follow-up questions
about being allowed to change shifts or how often respondents change shifts could be added for
those working a non-daytime schedule (i.e., Does your [employer/job] allow you to change the
shift you work? and/or How frequently do you change the shift you work?).
NEWScheduleNotice
How far in advance do you usually know your work schedule, that is, the days and hours
you work?
1.
2.
3.
4.
5.
6.
7.

One day or less
2 to 3 days
4 to 6 days
1 week
2 to 3 weeks
4 weeks or more
ALWAYS WORKS THE SAME SCHEDULE

Cognitive interviews

The same question wording was tested in Rounds 1 and 2 of cognitive interviews. The question
generally worked well in both rounds of testing, and no issues with proxy response were
observed. Several participants provided answers that mapped onto category 6, but then changed
their answer to category 7 during the debriefing, when they saw the categories. A couple

15

participants answered category 4 because they noted that their manager always released the
schedule weekly, even though their actual schedule (days and hours) never changed.
While most participants understood “work schedule” as the days and hours worked, some
participants (especially salaried workers) thought mostly about the meetings on their calendar or
the times they work, noting that everything else is very flexible. No major changes were
recommended for this question. The question was tested again in the web-probing survey.
Web-probing study

The same question wording was used in the web-probing survey.

One day or less

How far in advance do you usually know your work
schedule, that is, the days and hours you work? (n=448)
7.4%

2-3 days

8.9%

4-6 days

5.6%

1 week

13.4%

2-3 weeks

8.9%

4 weeks or more

7.1%

I always work the same
schedule

48.7%

Almost 50% of participants selected “I always work the same schedule.” This is likely a mode
effect because the response option was offered on-screen. In the cognitive interviews, a frequent
finding was that participants would change their response to “I always work the same schedule”
in debriefing upon seeing the category. This is unlikely to pose a problem unless it’s important to
collect the number of people who always work the same schedule. Alternatively, respondents
who say they always work the same schedule could be coded into the “4 weeks or more”
category. No other changes are recommended.

NEWTakeOff
How difficult would it be for you to take an hour or two off during work hours to take
care of personal or family matters?
1. Not difficult
2. Somewhat difficult
3. Very difficult

16

Cognitive interviews

The same version of this question was tested in Rounds 1 and 2 of cognitive testing. The
question worked well in both rounds, and there were no issues with proxy response. One
participant with low English ability answered this question in general, not in terms of during her
work hours. When asked about what “personal or family matters” meant to them, most
participants thought of medical appointments and childcare. Some people mentioned while they
CAN take time off during work hours for these reasons, they do not tend to do so since “it looks
bad” (to their employer). No wording changes were recommended for this question. The question
was tested again in the web-probing survey.
Web-probing study

The same version of the question was administered in the web-probing study.

Not difficult

How difficult would it be for you to take an hour or two
off during work hours to take care of personal or family
matters? (n=448)
65.0%

Somewhat difficult

29.2%

Very difficult

5.8%

Most participants indicated this would not be difficult or would be somewhat difficult. No
wording changes are recommended for this question.
SMJHome
As part of this job do you do any of your work at home?
1. Yes
2. No
Cognitive interviews

The same version of this question was tested in Rounds 1 and 2 of cognitive interviews. The
question worked well in both rounds and there were no issues with proxy response. However,
interpretations of the phrase “work at home” seemed to vary. Some participants thought of this as
ONLY work done in the home. Others included work done in coffee shops, libraries, or coworking locations. Some considered co-working locations to be on-site or office work, or
“remote work.” Participants mentioned their employers used terms like hybrid schedule, remote
work, and work at home. No wording changes were recommended for this question. The
question was tested again in the web-probing survey.

17

Web-probing study

The same version of this question was tested in the web-probing survey. About 2/3 of the sample
reported doing work at home. This is more than the most recent national average (close to 20%)
collected on the CPS, likely a byproduct of sampling participants from a nonprobability panel
who do surveys for pay.

Yes

As part of this job, do you do any of your work at
home? (n=490)
61.6%

No

38.4%

At the end of the web-probing study, all participants were asked a probe where they indicated all
of the places that they would consider to be part of “work at home.” This probe was administered
to gain insight into comprehension of the term “work at home.”

Working in my home

Some of the previous questions asked you about
“work at home.” Below is a list of locations where
people may work. Which of the following locations
would you consider to be “work at home”? Select all
that apply. (N=491)
96.5%

Working at co-working spaces

12.0%

Working in libraries

35.2%

Working in coffee shops

40.3%

Working from a friend/relative’s
home
Working at a client worksite

48.3%

Working from a hotel

38.7%

Working from a plane or train

23.4%

8.8%

As seen above, almost all participants selected “working in my home,” as expected. Of those
selecting this category, 36% ONLY selected “working in my home” and no other categories,
indicating these participants had the narrowest definition of work at home. The other participants
selected at least one additional category. The second most selected category was working from a
friend/relative’s home, followed by coffee shops, hotels, and libraries. This provides evidence
that most participants have a broader understanding of work at home than just working in one’s
own residence.
Final Recommendations

18

This question worked well in both the cognitive interviews and web-probing study. However,
participants had different conceptions of what “work at home” means. If a narrow definition of
work at home is needed, then interviewers may need to provide instructions on what counts as
“work at home” to read if necessary.
NEWCanHome/New12a
Could (you/NAME) work at home as part of [(your/his/her) main job/this job]?
1. Yes
2. No
Cognitive interviews (Rounds 1 and 2)

The same question wording was tested in both rounds. Although relatively few participants
received the question, those who did were able to answer it easily whether it applied to
themselves or another member of their household. Most people who received this question
answered “no” because the job took place in a physical location, like an ice rink or in a store.
One participant answering on behalf of their spouse said he couldn’t work at home because he
worked in intelligence and wasn’t able to access his materials remotely. Even if participants were
able to do some of their work at home (e.g., lesson planning), they didn’t consider that to be part
of their main job. One participant describing their spouse’s work at a physical education teacher
said, “[Spouse’s] main job is completely hands on with students. No, that could not change. So
no, he definitely could not work at home as part of his main job.”
Web-probing

A similar question was asked in the web-probing study. Close to 14% of participants indicated
that they COULD work at home as part of their job.
Could you work at home as part of this job? (n=189)
Yes

No

13.8%

86.2%

No wording changes are recommended.
NewCatchUp

19

On the days you work at home, do you work at home JUST to catch up on work or do
you work at home instead of traveling to a worksite? READ IF NECESSARY: Worksites
may include offices, factories, stores, warehouses, client locations, or even vehicles for truck
or other delivery drivers
1. Just to catch up or finish work
2. Home instead of traveling to worksite
3. Both
Cognitive interviews

The same version of this question was tested in Rounds 1 and 2 of cognitive interviews. No
major issues with proxy response were observed, however, some issues with comprehension
were observed.
Most participants understood this question as asking if they take work home to catch up on work
they could not complete in the office, versus if they did not want to commute to their office.
Other participants seemed to latch onto the word “travel” and focused primarily on the desire to
not have to commute, rather than working at home instead of working in an office or at a
worksite. One participant noted that catching up on work at home does not count as “working at
home” because it was work that should have been completed in the office. Several participants
thought the phrase “just to catch up on work” could mean that working onsite is too distracting,
and you choose to work at home to be able to catch up on work or writing tasks that need a lot of
focus.
In terms of response formation, some participants did not realize an answer of “both” could be
provided and wanted to change their answer to “both” during the debriefing. Participants
generally liked the “read if needed” definition of a worksite. Terms like worksite, offsite, and
onsite tended to resonate with most participants. Some participants expected to see co-working
spaces in the list of work sites. One participant said she would not include offices in the
definition – she thought more of remote locations instead.
Recommendations

We recommended either dropping this question due to inconsistencies in comprehension or
revising and retesting the question for the web-probing study to take emphasis off of the word
“travel.” For example:
Do you work at home JUST to catch up on work, or do you work at home instead of working at
an office or worksite?
We also recommended specifying whether co-working spaces should be included (either in the
definition itself, or as an interviewer instruction if asked).

20

Web-probing

The question was revised as follows for the web-probing to take emphasis off travel to a
worksite.
On the days you work at home, do you work at home JUST to catch up on work or do
you work at home instead of going to a worksite?
Worksites may include offices, factories, stores, warehouses, client locations, or even vehicles
for truck or other delivery drivers.
1. Just to catch up or finish work
2. Home instead of worksite
3. Both
Response distributions are shown below.

Just to catch up or finish
work
Home instead of worksite

On the days you work at home, do you work at home
JUST to catch up on work or do you work at home
instead of going to a worksite? (n=490)
17.4%
51.6%

Both

31.0%

Around half of participants reported working at home instead of at a worksite, and 31% said
“both.” This was likely a mode effect due to the categories being visible on the screen in the
web-probing study. As noted in the cognitive interview findings, participants sometimes wanted
to select “both” when they saw the category in the debriefing but did not realize that was an
option during the interview. It is likely that if the “both” category is read aloud by interviewers,
then a large share of participants will select that option.
Final Recommendations

This question had some measurement error in the cognitive interview study. Participants thought
that the question was forced-choice and did not realize that they could select “both.” This might
lead to overreporting in one of the response categories and not reflect hybrid workers who
regularly do both. Some participants also had different understandings of “just to catch up or
finish work,” which could mean finishing work they couldn’t complete in the office or choosing
to work at home since the office was too distracting. Similar information may be inferred from
the questions on whether respondents’ jobs are entirely at home, and the days and hours they
usually work at home. We recommend considering dropping the question due to the potential for

21

measurement error. If it’s included, consider whether the “both” category should be read aloud if
it’s analytically useful.
NEWDistance
(Do you/Does NAME) have the option to work on-site at this job?
1. Yes
2. No
Could (you/NAME) regularly travel to (your/his/her) [employer’s] worksite instead of
working from home if (you/he/she) wanted to?
1. Yes
2. No, couldn’t travel
3. No [employer] worksite to go to
Cognitive interviews (Round 1 and 2)

The same version of the question was tested in both rounds. Participants who said they worked at
home were asked about both above items during testing. Participants tended to understand the
first question in this series well for both themselves and other household members. On-site was
often interpreted as office locations or client offices. For others, the term “on-site” seemed to
encompass any physical space at which work could be completed. For example, an architect said,
“on-site for my job is the construction site.” Another participant said their husband’s employer
provided access to co-working spaces such as WeWork but wouldn’t consider that to fall under
on-site because “it’s not like they have one WeWork that everyone goes to. It’s all based on
convenience.” Overall, most participants defined on-site as being a physical space associated
with their employer where they could work, whether they wanted to or was feasible.
Several participants answered “no” to this question because there was no physical space for them
to go to. Others answered “no” because, although their employer did technically have an office
space, the participants’ role was as a remote worker so there was no physical space for them.
Even though these participants could commute, they still answered “no” because of the lack of
space. One participant said “no” because the on-site location was in another city. This
participant, then, answered based on the impracticality of traveling to the space rather than the
existence of the space. Another participant – an independent consultant – said their answer could
change depending on their client roster and whether a client allowed them to work at the office.
This participant answered “no” because their only client was a tennis instructor who didn’t have
a physical workspace, but acknowledged that future client could offer office space, thus changing
their answer.
The alternative wording in the second question caused more difficulty for participants. Because
the response options weren’t read aloud, participants didn’t know to specify whether their “no”
response indicated their inability to travel or because there was no employer worksite to go to.
This led to the interviewer having to probe between “no, couldn’t travel” and “no employer
22

worksite to go to.” For participants who had an employer worksite but lived too far from the
office, making the commute impractical, still answered “no employer site to go to.” One
participant said, “That one is more confusing. I guess I’d say ‘no employer worksite to go to.’
Travel isn’t the problem, the building does exist, there’s not space for me. I could get to the
worksite. I would probably answer #3, but it wouldn’t be a great answer since there is a building
I could go to.” This response indicates that there may not be a meaningful distinction between
categories 2 and 3 and the question is more burdensome for participants whose role is remote,
has a worksite, and lives within a commutable distance to the office building.
The definition of “travel regularly” differed significantly between participants. Some participants
perceived the question as asking about the practicality of traveling regularly (e.g., “There’s no
office in the city that I live in. No physical office.”) and living in a commutable distance to the
office space (e.g., “If someone lived 2 hours away they couldn’t drive regularly”). When probed
on the conditions that must be met for someone to regularly travel to a worksite, participants’
responses differed. Some participants said that someone would have to be able to drive rather
than fly. For instance, one participant in the DC area said, “The ease of being able to get to the
place. I was thinking if our headquarters was in Colorado and they said, ‘You could work onsite.’ I wouldn’t consider that as being able to regularly travel. It would need to be a daily
commute that makes practical sense. You could do it without having to fly somewhere.”
Participants also varied in how frequently one must commute to and from a worksite for it to be
considered regular travel. Some participants said regular travel occurred as frequently as 4-5
times per week while others felt a couple times or once a week would suffice. One participant
thought that once a quarter by plane would count as regular travel.
All but two participants said these two questions were asking about different behaviors.
Participants tended to view the first question as asking about whether the option to work on-site
exists while the second question asks about whether one could work on-site, could one do it on a
regular basis. One participant said, “They are different. One asks if it’s possible and two asks if
it’s practical.”
Recommendations

It was recommended that the first question was kept for additional testing and the second
question is dropped. If the distinction between not working onsite because of distance versus a
worksite not existing is important, a wording change may be needed. Some new wording
suggestions included:
• At your (main) job, does an employer worksite exist?
o [IF YES] Do you have the option to work on-site at this job?
• Is there an employer worksite to go to?
o [IF YES] For what reasons do you not work there?
▪ Can’t travel; Too far away; Not in a commutable distance; No space
available; Prefer not to work onsite; Other (please specify)

23

Web-probing

The question was revised to “Is there a worksite you could go to?” in the web-probing study.
Prior to answering this question, web-probing participants were first asked how frequently they
work at home. Response distributions are shown below.
How frequently do you work at home? (n=302)
Job is entirely at home

39.4%

At least once a week

46.7%

At least once every two weeks

7.0%

At least once a month

4.6%

Less than once a month

2.3%

Almost 40% of the web-probing participants who work at home also reported being remote
workers, that is, their job is entirely at home. Almost half of participants (47%) reported working
at home at least once a week. Participants who indicated they were remote workers were asked
the question about whether there is a worksite they could go to. Most (63%) of participants
indicated there was not a worksite they could go to.
Is there a worksite you could go to? (n=119)
Yes

No

37.0%

63.0%

Web-probing participants who indicated that they could work onsite but do not (n=44) were first
asked to provide an open-ended description of the MAIN reason they do not work onsite. Most
of the responses fell into the categories provided in the subsequent closed-ended version of the
question. For example, participants mentioned things like flexibility, preference, or avoiding a
long commute. Some participants mentioned that the worksite is empty and it wouldn’t be worth
it to travel to an empty office. A few participants said it was simply because they are “not
required” to work on-site and they do not go to the worksite because they are “allowed” to work
at home.
Response distributions to the closed-ended version of the question are shown below. No
participants selected “Other,” indicating the provided categories were sufficient.

Live too far away

What is the MAIN reason you do not work there?
(n=44)
13.6%

Prefer not to commute or lack
transportation

11.4%

24

No space/resources available at
worksite
Health reasons

4.5%

Flexibility, work-life balance,
personal or family needs
Prefer working at home, get more
done at home
Other reason

34.1%

2.3%

34.1%
0.0%

Participants who indicated they did not work at home were asked to provide the main reason. As
expected, the most selected choice was that the job can’t be done from home.

Job can’t be done from home

What is the MAIN reason you do not work
at home? (n=189)
73.0%

Not interested/personal preference

4.2%

Child care or family conflicts

1.6%

No equipment to work remotely/no space

3.7%

More productive at work, better connection
with coworkers
Loss of opportunity, income, or promotion;
manager doesn’t support
Other reason

7.9%
4.2%
5.3%

A total of 10 participants selected ‘Other reason.’ Most of the reasons provided could have been
coded to the closed-ended categories (e.g., manager doesn’t support, job can’t be done from
home). The question series used in the web-probing study seemed to work well; no additional
wording changes are recommended.
SMJHomeHours, SMJHomeDays, SMJHOnlyDays
When you work at home, how many hours per week do you USUALLY work at home
for this job/business? ___ hours
How many days a week/every two weeks do you USUALLY work at home on this
job/business? ___ days
How many of these days do you work ONLY at home, that is, days when you don’t travel
to the office or other work site? ___ days

25

Cognitive interviews

This series of questions collected the hours and days usually worked at home, and of those days,
how many participants ONLY work from home. The same version of this question was tested in
Rounds 1 and 2 of cognitive interviews. Some minor issues with self and proxy response were
observed where participants did not seem confident in their estimates of the exact number of
days they or other household members work at home. Some participants mentioned that this
question was sometimes difficult to answer in the context of the other questions, and that they
were distracted by trying to make sure their responses were consistent with one another. For
example, some participants mentioned that the hours and days in which they work at home can
vary somewhat:
“There are some weeks when work at home 4 days a week then go into the office once a
week.”
A couple of participants felt it varied so much that that they wanted to give a range and were
prompted to try to provide a single number. One participant answered based on the policy her
workplace has in place (work at home 3 days, come into the office 2 days) rather than the
number of days she had been working at home recently. She wondered if the question should
have a reference period or ask about the typical or average amount of time. Some participants
noted that the questions on the number of hours or days they work at home could be dropped, as
they felt redundant.
Recommendations

We recommended including a prompt to estimate or provide an average if people have a difficult
time answering, for example:
Read if necessary: If the number of days varies, provide your best estimate; In a typical week,
how many of these days…
In addition, we recommended considering dropping some of the questions on hours and days
worked at home to reduce perceived redundancy.
Web-probing study

This series of questions was tested again for the web-probing study. These questions were asked
for those who indicated they work at home (but were not remote workers), and to those
indicating they work at home at least once per week or once every two weeks. The median hours
usually worked at home was 10 hours, the median number of days a week usually worked at
home was 2 days, and the number of days only worked at home was also 2 days.
Median reported
Number of hours usually worked
at home per week (n=187)

10 hours
26

Number of days a week usually
worked at home (n=163)
Number of days worked ONLY at
home (n=163)

2 days
2 days

Web-probing participants were then asked to report the days they work only at home.

0

How many of these days do you work ONLY at home, that is,
days when you don’t travel to the office or other work site?
(n=163)
5.5%

1

32.5%

2

39.3%

3

16.0%

4

3.7%

5

2.5%

6

0.6%

A total of 18 participants reported values for SMJHOnlyDays that exceeded those reported in
SMJHomeDays. Because SMJHOnlyDays is supposed to represent the share of “those” days
usually worked only at home, this suggests those participants may not have understood the
question, provided inconsistent responses, or provided rough estimates of the days usually
worked at home versus only at home that were internally inconsistent.
Final recommendations

Cognitive interview participants had some confusion answering this series of questions about
working at home, when they were providing estimates to earlier questions about the number of
days and hours that they usually work at home. Participants tried to make their answers
consistent with one another and would sometimes get tripped up doing so. Some web-probing
participants also gave inconsistent responses. We recommend dropping SMJHomeOnlyDays
because the two previous questions asked for USUAL hours. However, SMJHomeOnlyDays asks
for the share of days reported in the previous questions, which was based on usual hours. Usual
hours were sometimes estimates, making it difficult to provide consistent responses to the
question.
SMJHomeReason
Cognitive interviews

27

Round 1

What is the MAIN reason why (you/NAME) (work/works) at home?
1. Finish or catch up on work
2. Business is conducted from home
3. Nature of the job
4. Coordinate work schedule with personal or family needs
5. Reduce commuting time or expense
6. Personal preference
7. Some other reason
Although most participants understood what this question was asking and could provide
responses for themselves, their responses rarely mapped onto one of the 6 main reasons, resulting
in many “other” responses. At least one participant had trouble identifying the main reason for
another household member. These participants tended to say their main reason for working at
home was because of convenience or flexibility, neither of which perfectly mapped onto one of
the prescribed categories. For example, one participant said, “Flexibility. I have younger kids so I
like the flexibility.” Another participant said their main reason was to “handle personal tasks.” A
language tutor who had clients come to their home said, “I have a lot of materials that each child
needs, so it’s easier to have them come to me instead of travelling from place to place.
Additionally, participants often gave multiple reasons for working at home, requiring the
interviewer to probe for the one main reason. A participant providing a single clear reason for
working at home rarely occurred.
When shown the response categories during the debriefing period, participants noted that
“personal preference” encompassed several reasons and was a good catch-all. Others mentioned
that they would have answered differently if they saw the response options. Some participants
said they would have selected “personal preference” while others said they would have selected
“nature of the job.”
Several participants had difficulty answering the question because – even after seeing the
response options – they wanted to select multiple responses. One participant said, “Just one of
these? I would probably say 2.” When probed, the participant followed up, “I mean, they’re
clear. But I think it’s one of those things where there’s overlapping options. It does say the main
reason so…You could work at home one full day to reduce commuting time or expense, but also
there’s stuff that needs to get done at home so you could work at home to coordinate your work
schedule with family/personal needs.”
In another example, one participant worked at home because their company went virtual during
the pandemic, but they also prefer to work at home, thus complicating their response process.
Some participants were unsure of the difference between “business is conducted from home” and
“nature of the job.” Self-employed participants gravitated towards “business is conducted from
home” whereas wage and salary workers felt that “nature of the job” was more appropriate.
Round 2

28

The question was revised as follows in Round 2:
What is the MAIN reason why you work at home?
1. Finish or catch up on work
2. Job is remove, there is no other worksite
3. Expected to work at home, that’s the nature of the job
4. Business is conducted from home
5. Coordinate work schedule with personal or family needs
6. Reduce commuting time or expense
7. Personal preference
8. Some other reason
In Round 2, an option was added (i.e., Job is remote, there is no other worksite) and option #3
(i.e., Nature of the job) was modified (i.e., Expected to work at home, that’s the nature of the job)
to capture people working at home because their employer went virtual or because they work a
hybrid schedule. Similar problems observed in Round 1 were also observed in Round 2.
Responses given did not map onto the response categories. During the interview, participants
mentioned several reasons that could not easily be mapped to a response category. Moreover, the
interviewer often had to probe for a single, codeable response. Several participants suggested
that the response format be select-all-that-apply because it was too difficult to indicate a main
reason for working at home. Participants interpreted option 2 as applying to individuals whose
employer was completely virtual while option 3 would be more appropriate for hybrid workers
or workers whose employer had a physical office but weren’t required to work in that office. For
instance, one participant described the difference between option 2 and 3 as “[option 2] being no
central building where people are gathering but [option 3] is there is a central building but
someone like me can’t go there because there is no space.”
Recommendations

Participants had difficulty pinpointing the difference between options 3 and 4. If the intention is
to determine whether a self-employed respondent run their business out of their home, it is
suggested that option be displayed for business owners only to minimize the confusion. Is it also
suggested that additional reasons for working remotely be added to option 2. Specifically, remote
workers’ employers may have a physical space but some workers may not have the option to
work in that space because of an impractical commute or a lack of space. Similarly, option 3 is
double-barreled, potentially leading to confusion, especially for hybrid workers. Some jobs may
have schedules that require their employees to work at home some days of the week, but that
isn’t the nature of the job. Separating these categories could alleviate some respondent confusion.
Finally, providing instructions for how to code responses such as “flexibility” or “convenience”
to interviewers could reduce the need to probe for a codeable response and minimize error.
Below are suggestions for response options:
• What is the MAIN reason why you work at home?
1. Finish or catch up on work
2. Job is remote (no worksite, cannot travel to worksite, or no space at worksite)
29

3. Expected to work at home some or all of the week
4. Business is conducted from home [BUSINESS OWNERS ONLY]
5. Coordinate work schedule with personal or family needs
6. Reduce commuting time or expense
7. Personal preference
8. Some other reason
Web-probing

Web-probing participants who indicated they worked at home were asked an open-ended
question about the MAIN reason why they work at home. Most of the responses had to do with
the convenience and flexibility of being able to work at home, work-life balance, or avoiding a
long commute. Several mentioned that work at home was adopted during the pandemic and has
continued since then; these participants mentioned they now have a hybrid schedule or work
remotely. Several also mentioned catching-up on work at home or being able to focus better
when at home. A SurveyMonkey-provided WordCloud of the open-ended responses is displayed
below.

The distribution of responses to the closed-ended question are displayed below. Like the
cognitive interviews, most participants (over 35%) selected reasons pertaining to flexibility and
convenience. Interestingly, this exceeded selections of personal preference (14.8%), which was
frequently selected in the cognitive interviews. Reducing commuting time (18.7%) and finishing
or catching up on work (15.9%) were the next most frequently selected options.

Finish or catch up on work

From the list below, what is the MAIN reason why
you work at home? (n=182)
15.9%

Job is remote, no worksite, no
space/resources at worksite
Job requires working from home

1.1%
4.4%

30

Reduce commuting time/expense
or lack transportation
Health reasons

18.7%

Flexibility, work-life balance,
personal or family needs
Personal preference, get more
done at home
Other reason

35.2%

3.3%

14.8%
6.6%

A total of 12 ‘Other’ responses were provided. Some of these could be coded to the existing
categories (e.g., reduce commuting, no resources at worksite).
•
•
•
•
•
•
•
•
•
•
•
•

Location of customers closer to my home than work
I am expected to basically be on call for any issues that occur outside of normal business
hours.
Opened half day on Friday and the boss said I could work from home on that day.
Computer is available at home, but not on site.
Training
When Covid hit, my supervisor allowed me to start work at home when needed.
It wasn't my choice.
I work in a laboratory chiefly, but can work from home to process data.
What I said on the last screen
From time to time I'm given special projects that can be done from home.
Job is hybrid
Contingent on my boss' mood, health condition, situation

Final Recommendations

The new categories tested in the web-probing study seemed to perform well. Nearly all the openended responses could be mapped back to the provided categories. Interviewers may need some
additional training on how to code responses that aren’t immediately obvious (e.g., if a
respondent says “my job is hybrid”). In those cases, interviewers may need to read the categories
and ask respondents to select one, or these may get coded into ‘Other.’
NonCompete
Some employers try to restrict what their employees can do after they leave their job.
For your main job, did you agree that if you left your employer, you would not start or
join a competing business? This is often called a non-compete agreement.
1. Yes
2. No

31

Cognitive interviews

The same version of this question was tested in Rounds 1 and 2 of cognitive testing. The
question wasn’t understood well and needed to be repeated for several participants. Some
participants also weren’t sure if they signed one, especially for proxy response. For example, a
few participants noted there were rules about what they can or can’t do when they leave the job
but were not sure if these rules were part of a non-compete agreement. One participant
suggested removing the first sentence of the question to avoid confusion on what to include for
this question, and that the last sentence could also be removed. The question was lengthy to read,
and as mentioned, some participants needed it repeated because the actual question came in
between two other sentences.
Recommendations

We recommended testing a simplified version for the web-probing study, for example:
For your main job, did you agree that if you left your employer, you would not start or join a
competing business?
Web-probing

This question was revised for the web-probing study and retested. Participants were randomly
assigned to receive one of the following versions of the question. All participants also received a
follow-up probe asking for their level of confidence in their response to this question.
Version A: Some employers try to restrict what their employees can do after they leave their
job. For your main job, did you agree that if you left your employer, you will not start or join
a competing business? This is often called a non-compete agreement.
1. Yes
2. No
Version B: For this job, did you sign a non-compete agreement that restricts you from starting
or joining a competing business if you left your employer?
1. Yes
2. No
How confident are you in this response?
1. Not confident
2. Somewhat confident
3. Very confident

32

Response distributions by version are displayed below (n=447). Most participants (~85%)
indicated that they had not signed a non-compete agreement. Chi-square analyses indicated there
was no difference in responses based on question version (p = 0.49).
Yes

No

Version A (n=220)

15.5%

84.5%

Version B (n=227)

13.2%

86.8%

Participants answered a follow-up probe on their level of confidence in their response. Overall,
most participants (81.0%) indicated that they were very confident, 18.3% were somewhat
confident, and 0.7% were not confident in their response. A chi-square analysis indicated that
there was no difference in confidence based on question version (p = .17).
Final Recommendations

We recommend dropping this question due to poor comprehension, difficulty with proxy
response, and lack of confidence in responses in both the cognitive interviews and web-probing
study. The question also does not fit well with the topic of the survey, on work flexibilities and
work at home. If the question is kept, a simplified version may be considered: For your main job,
did you agree that if you left your employer, you would not start or join a competing business?
This is often called a non-compete agreement.
NEWStop, NEWStopNum, NEWStopWHY
At any time IN THE LAST 3 MONTHS, were (you/NAME) told by (your/his/her)
employer to stop working after (you/he/she) had already started because there was no
need for (your/him/her) to work?
READ IF NECESSARY: For example, being sent home because there were not enough
customers in the store or due to a shortage of supplies.
1. Yes
2. No
How many times IN THE LAST 3 MONTHS did this happen?
1. ______(numerical response)
What was the MAIN reason there was not enough work for you to do?
1. Lack of customers
2. Lack of supplies or input material
3. Weather-related
4. Performance not adequate
5. Something else

33

Cognitive interviews (Rounds 1 and 2)

The same version of this question was tested in both rounds. Participants were asked for their
feedback on the question during debriefing, regardless of whether the whole series was
administered during the interview. Overall, this question was not well understood. Many
participants needed the question to be repeated and some only understood what the question was
asking upon seeing the response options in NEWStopWhy. Three of the 18 participants probed
on this question thought it referred to being laid off. For example, one participant said,
“…someone is hired and there’s not enough need for their labor anymore. Whether it’s physical
products to sell or services to provide. They’re no longer needed.” Another participant said,
“This is for someone who’s in a job that depends on volume, say, construction. So if they are a
sheet metal worker and the company didn’t get enough contracts for building residential
property, they let people go because they couldn’t guarantee them enough hours.”
Most of the confusion stemmed from all the participants being salaried workers and not ever
expecting a work stoppage to happen to them. Along those lines, many participants indicated that
this question seemed to apply to non-salaried workers, shift workers, or hourly workers. One
participant who was a salaried worker said, “That in my mind doesn’t really relate to my type of
work. That relates more to someone who does shift work or seasonal work. Like, if I worked in a
restaurant and no one was there and they said, ‘Hey, we don’t need you. You can go home.’”
A salaried worker wondered if this question could cover being asked to stop working on a
project: “It doesn’t make sense to me. Are you saying stop working on a specific project or you
have no more work to do period? I don’t understand at all. It could mean ‘Don’t work for the day
or you’re getting laid,’ or ‘Stop working on a specific project.”
Participants generally understood NEWStopWhy, when the response categories were visible in
the debriefing. Some participants commented that option 4 (i.e., Performance not adequate)
seemed out of place and sensitive and wondered if people would admit they were told to stop
work because of inadequate performance. Others wondered if option 4 referred to being laid off
completely rather than asked to leave early. One participant was unsure of the meaning of “input
material”. “Weather-related” stood out to a few participants but, upon further reflection,
participants thought having to stop work because of weather could be plausible.
Web-probing

A revised version of this question was tested in the web-probing study. Most participants (over
90%) indicated that they had not been told to stop working in the last 3 months. Participants
responding yes (8.1%) were asked for the number of times this happened. The average reported
was 3 times, with a range from 1-10 times.
At any time IN THE LAST 3 MONTHS, were you told by your employer to stop
working after you had already started your workday because there was no need for

34

you to work? For example, being sent home because there were not enough customers
or due to a shortage of supplies. (n=447)
Yes
No
8.1%
91.1%
Participants were asked about the main reason there was not enough work for them to do.
Response distributions are shown below. Note that the example for the first question was
displayed on the screen, and the latter two questions (NEWStopNum and NEWStopWHY) were
displayed together on the same screen, which likely increased comprehension in the web-probing
study.

Lack of customers

What was the MAIN reason there was not enough
work for you to do? (n=40)
62.5%

Lack of supplies or input materials

12.5%

Weather-related

10.0%

Performance not adequate

2.5%

Something else

12.5%

Final Recommendations

Participants in the cognitive interview study did not understand this question well. We
recommend dropping the question as it’s unlikely to be well-understood in an intervieweradministered mode. The question is also lengthy and dropping it will help decrease respondent
burden. If the question is kept, the examples should be read aloud and not read if necessary
instructions.

Second jobs
Only one participant in the cognitive interview study worked more than one job. Thus, analysis
of the question series on second jobs is restricted to web-probing participants. A total of 114
web-probing participants (23.4% of participants) reported having more than one job. These
participants were administered a series of questions about their second job, that is, the one in
which they work the second most hours.
Participants reported that they usually work a median of 3 days per week on their second job.
The days of the week worked on second jobs varied more than those worked on participants’
main jobs (which was mostly 5 days per week, Monday-Friday), and were evenly distributed
across the week.
35

SSJDays (SSJ1A)

Sunday

Which days of the week do you USUALLY work on this job?
Choose all that apply. (n=114)
11.0%

Monday

12.6%

Tuesday

11.8%

Wednesday

11.8%

Thursday

13.4%

Friday

13.6%

Saturday

14.3%

SSJHome (SSJ2)

Web-probing participants were asked if they do any of their work at home for their second job.
About two-thirds of participants indicated they work at home for their second job.
As part of this second job, do you do any of your work at home? (n=116)
Yes

No

67.2%

32.8%

SSJHomeBus (SSJ3)

Web-probing participants were asked if the work they do for their second job is for a home-based
business. Most participants (around 56%) indicated their second job is for a home-based
business.

Is the work you do at home as part of this job for a home-based business? (n=78)
Yes

No

56.4%

43.6%

SSJHFreq (SSJ4)

36

Web-probing participants were asked how frequently they work at home. Most participants
(82%) reported that their second job is done entirely at home.
How frequently do you work at home? (n=78)
Job is entirely at home

82.1%

At least once a week

9.0%

At least once every two weeks

5.1%

At least once a month

2.6%

Less than once a month

1.3%

NEWSJDistance

Most participants (over 90%) indicated that there is not a worksite they could go to for their
second job.
Is there a worksite you could go to? (n=64)
Yes

No

9.4%

90.6%

SSJHomeHours (SSJ5)

Participants indicated that they work an average of 6.3 hours per week as part of their second
job, with a range from 1 to 20 hours per week.

SSJReasonMJ (SSJ6)

Web-probing participants were asked to provide the main reason they worked multiple jobs;
response distributions are displayed below.

Meet expenses or pay off debt

What is the MAIN reason that you worked at
more than one job? (n=116)
37.9%

Earn extra money

49.1%

Build a business or get experience in
a different job
Enjoy the second job

4.3%

Other reason

1.7%

6.9%

37

Most participants indicated they worked a second job for financial reasons (e.g., to earn extra
money or to meet expenses or pay off debt). Although the first two categories were similar, there
was a good distribution across each of them. Based on these finidngs, no changes are
recommended to the series on second jobs.

38