Attachment D Online Protocol

ORS - Cog Items Test, Summer 2017

Before continuing, please enter the two pieces of information requested below. If you supervise workers in
more than one job, please pick one.
1. The name of your current job (leave blank if currently not working).
2. The name of the job you currently supervise (or supervised).
Name of Your Current Job

Name of the Job You Supervise(d)

ORS - Cog Items Test, Summer 2017

You will be asked to answer some questions starting on the next page. When answering
these questions, please think about the job: {{ Q1.R2 }}.
Do not answer the questions about your own job.
As you answer the survey questions, please point out anything that is vague or confusing
about the question or possible answers.
If space is provided, type in your answers. Otherwise, you can just give your answer
verbally.
There are 13 questions, but many have yes or no answers, so don't feel like you have to
rush.
Click Next below to see the first survey question.

ORS - Cog Items Test, Summer 2017
Work Pace

1

Reminder: You are answering questions about the job: {{ Q1.R2 }}.
Question 1
Which of the following most determines the pace of work for this job? (select one)
Machinery, equipment, or software
Numerical performance targets or quotas enforced by company
Other persons (e.g., co-workers, supervisors, customers, members of the public) determine pace
Self-paced by worker within general performance guidelines
Other external source

ORS - Cog Items Test, Summer 2017
Work Flow

Question 2
How would you describe the work flow for this job? Would you say that it ….
Has few, if any, slack periods
Has few, if any, rush periods (or large build-ups of work)
Varies, with fluctuations of rush periods (or large build-ups of work) and slow periods on daily or weekly basis

ORS - Cog Items Test, Summer 2017
Breaks

2

Reminder: You are answering questions about the job: {{ Q1.R2 }}.
Question 3
Can workers step away from their work area easily outside of scheduled breaks (for example, for short
bathroom breaks)
Yes
No

ORS - Cog Items Test, Summer 2017
Review of Work

Question 4
How frequently is work checked in this job?
More than once per day
Once per day
At least once per week, but not every day
Less than weekly

Question 5
Are supervisors generally present in the same work area as workers?
Yes
No

ORS - Cog Items Test, Summer 2017
Problem Solving

3

Reminder: You are answering questions about the job: {{ Q1.R2 }}.
Question 6
The next question is about “problem solving” tasks that the worker does in this job. Think of “problem
solving” as what happens when workers are faced with a new or difficult situation which requires them to
think for a while about what to do next. How often is the worker responsible for solving problems that take
more than 5 minutes to find a good solution?
More than once per day
Once per day
At least once per week, but not every day
Less than weekly, but at least once per month
Less than monthly, including never

ORS - Cog Items Test, Summer 2017
Personal Contacts

Question 7
How often does this job require verbal, work-related interactions?
Constantly, every few minutes
More than once per hour, but not constantly
More than once per day, but not more than once per hour
No more than once per day, including never

Question 8
What level of “people skills” does this job require?
Minimal/Basic
More than basic

ORS - Cog Items Test, Summer 2017
Yes/No Questions

4

Are workers in this job ...
Required to work with the general public?
Yes
No

Required to work around crowds?
Yes
No

Required to supervise others as part of job?
Yes
No

Permitted to work from home or telework?
Yes
No

Required to work outside?
Yes
No

ORS - Cog Items Test, Summer 2017
A Few Follow-up Questions

5

Those are all the questions. Now that you've had a chance to answer each one, we would
like to go through a few of them one more time, and ask you a few clarifying questions to
help us better understand how you interpreted the question.
It's possible that you have already answered some of the questions we are going to ask
through your comments when you initially answered a question. If so, just skip the
question.
In this review, all of your answers will be verbal. You don't have to type anything.
Finally, take your time. Don't worry if you run out of time.
Click Next to begin.

ORS - Cog Items Test, Summer 2017
Review of Question 1

The first question you were asked is shown in red below, followed by the answer that you selected.
The answers that you did not select are shown directly above Question 1. Please answer the questions
below that appear in blue text.
Did you have any difficulty picking one factor that most controls the pace of work?
Were any of the descriptions of factors that drive the pace of work unclear to you? If so, which ones?
What does the phrase "Self-paced by worker within general performance guidelines" mean to you?
Following are the answers that you did not select for Question 1:
Machinery, equipment, or software
Numerical performance targets or quotas enforced by company
Other persons (e.g., co-workers, supervisors, customers, members of the public) determine pace
Self-paced by worker within general performance guidelines
Other external source

6

Question 1
Which of the following most determines the pace of work? (select one)
Machinery, equipment, or software
Numerical performance targets or quotas enforced by company
Other persons (e.g., co-workers, supervisors, customers, members of the public) determine pace
Self-paced by worker within general performance guidelines
Other external source

ORS - Cog Items Test, Summer 2017
Review of Question 2

The second question, Question 2, that you were asked is shown below in red.
When you heard the phrase "work flow," what did you think of?
The intent of this question is to capture the pace of work on the job. How well does this question capture
the pace of work?
Was any part of the question confusing?
Question 2
How would you describe the work flow for this job? Would you say that it ….
Has few (if any) slack periods
Has few (if any) rush periods (or large build-ups of work)
Varies, with fluctuations of rush periods (or large build-ups of work) and slow periods on daily or weekly basis

ORS - Cog Items Test, Summer 2017
Review of Question 3

7

The third question that you were asked is shown below in red.
In your own words, what do you think this question is asking?
Question 3
Can workers step away from their work area easily outside of scheduled breaks (for example, for short
bathroom breaks)?
Yes
No

ORS - Cog Items Test, Summer 2017
Review of Question 4

The fourth question that you were asked is shown below in red.
What does it mean to have work checked in the job of {{ Q1.R2 }}?
Question 4
How frequently is work checked in this job?
More than once per day
Once per day
At least once per week, but not every day
Less than weekly

ORS - Cog Items Test, Summer 2017
Review of Question 5

8

The fifth question is also shown below in red.
When you answered this question, how did you define "being in the same work area?"
Would you consider a supervisor who was in the same building, but not in the same room, as being in the
same work area?
Question 5
Are supervisors generally present in the same work area as workers?
Yes
No

ORS - Cog Items Test, Summer 2017
Review of Question 6

Question 6 is shown below in red text.
Please explain how you came up with your answer to this question.
Question 6
The next question is about “problem solving” tasks that the worker does in this job. Think of “problem
solving” as what happens when workers are faced with a new or difficult situation which requires them to
think for a while about what to do next. How often is the worker responsible for solving problems that take
more than 5 minutes to find a good solution?
More than once per day
Once per day
At least once per week, but not every day
Less than weekly, but at least once per month
Less than monthly, including never

ORS - Cog Items Test, Summer 2017

9

Question 7 is shown below in red text.
Can you think of a different way to ask this same question?
Question 7
How often does this job require verbal, work-related interactions?
Constantly, every few minutes
More than once per hour, but not constantly
More than once per day, but not more than once per hour
No more than once per day, including never

ORS - Cog Items Test, Summer 2017

Question 8 is shown in red text below.
How would you define minimal or basic "people skills?"
Question 8
What level of “people skills” does this job require?
Minimal/Basic
More than basic

ORS - Cog Items Test, Summer 2017

You are done. Thank you for your help!

10