Form Attachments Survey Protocol

Attachment A

Revised ORS Cognitive Elements Items

Introduction

Unlike some tasks you’ve completed on TryMyUI, we’re not looking for you to evaluate a website, rather we’re looking for you to answer a series of questions and give us your written and verbal feedback. Please talk aloud, giving your reactions and thoughts as you move through the pages.

This is a survey for the US Bureau of Labor Statistics. This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141 (Expiration Date: April 30, 2018). This survey will take approximately 20 minutes to complete. You are not required to respond to this survey unless it displays a currently valid OMB control number.

We will use the information you provide for statistical purposes only. Your participation is voluntary and you have the right to stop at any time. This survey is being administered by SurveyMonkey and resides on a server outside of the BLS Domain. The BLS cannot guarantee the protection of survey responses and advises against the inclusion of sensitive personal information in any response. By proceeding with this study, you give your consent to participate in this study.

We are looking for information about how respondents answer our surveys. Please take your time as you answer these questions. The information you provide will contribute to valuable research at the BLS, one of the leading federal statistical agencies.

Control of Work Pace

Which of the following most determines the pace of work? (select one)

□ Machinery, equipment, or software controls pace

□ Numerical performance targets or quotas enforced by company determine pace

□ Other persons determine pace (e.g., co-workers, supervisors, customers, members of the public)

□ Other external source (specify) ___________________

□ Self-paced by worker within general performance guidelines

Work Flow

How would you describe the work flow for this job? Would you say that it ….

□ Has few (if any) slack periods

□ Has few (if any) rush periods (or large build-ups of work)

□ Varies, with fluctuations of rush periods (or large build-ups of work) and slow periods on a daily or weekly basis

Pause Control

Can workers step away from their work area easily outside of scheduled breaks (e.g., short bathroom breaks)

□ Yes

□ No

Work Review

How frequently is work checked in the occupation?

□ More than once per day

□ Once per day

□ At least once per week, but not every day

□ Less than weekly

Are supervisors generally present in the same work area as workers?

□ Yes

□ No

Problem Solving

The next question is about “problem solving” tasks that the worker does in his/her job. Think of “problem solving” as what happens when workers are faced with a new or difficult situation which requires them to think for a while about what to do next. How often is the worker responsible for solving problems that take more than 5 minutes to find a good solution?

□ More than once per day

□ Once per day

□ At least once per week, but not every day

□ Less than weekly, but at least once per month

□ Less than monthly, including never

Personal Contacts

How often does this occupation require verbal, work-related interactions?

□ Constantly, every few minutes

□ More than once per hour, but not constantly

□ More than once per day, but not more than once per hour

□ No more than once per day, including never

What level of “people skills” does this job require?

□ Minimal/Basic

□ More than basic

Checklist Items

Are workers in this occupation:
Required to work with the general public	Yes No
Required to work around crowds	Yes No
Required to supervise others as part of job	Yes No
Permitted to work from home or telework	Yes No
Required to work outside	Yes No

Attachment B – Protocol for In-house Cognitive Testing

• Hi! Thank you for coming in today.
  
  

• [Respondent reads and signs consent form.]

• [If team member is observing] I have a colleague in the next room that will be observing and taking notes.

• Today we’re going to be evaluating questions for a new survey that collects information about the demands of occupations.

• When this survey is conducted, we collect information about jobs by speaking to supervisors, managers, or the human relations staff in a company. Today I’m going to be asking you questions about the mental or cognitive demands of a job that you supervise. For example, the questions will ask about things like what is the pace of work and what determines the pace, how closely supervised is the position, who does the worker interact with, and so on.

• Since you’ve been a supervisor, we want to ask you the questions about a job that you supervise.

• So, to begin, can you tell me the title or name of the job that you supervise?

• Title/Name of Job supervised: ___________________

• How long have you supervised this job? __________

• Were you responsible for assigning some type of an annual job performance rating?

  • Yes

  • No (if no – make sure he/she had other supervisory responsibilities such as assigning work, monitoring progress, giving feedback, etc.)

• I want to assure you that all the information we discuss today will be kept completely confidential, and you won’t be identified in any reports that we prepare. So please read and sign this consent form, and let me know if you have any questions. Notice that we want to record the session. I do that just to help me with note taking.

• Before we begin, I want to emphasize that we’re not here to evaluate you or how good a supervisor you are. We’re only interested in how well the questions work. For example, are the questions clear and easy to understand?

• If a question isn’t clear or doesn’t make sense, please tell us. You’re not going to hurt our feelings. These questions are going to be asked all around the country with thousands of people so they need to be clear and easy to understand.

• Do you have any questions before we begin? I’m going to start the tape recorder now.

• Again, these questions are being asked about the job [title/name] that you supervise. Here’s the first question.

1. Which of the following most determines the pace of work on this job? (Select one. Show card with response options)

□ Machinery, equipment, or software controls pace

□ Numerical performance targets or quotas enforced by company determine pace

□ Other persons determine pace (e.g., co-workers, supervisors, customers, members of the public)

□ Other external source (specify) ___________________

□ Self-paced by worker within general performance guidelines

2. How would you describe the work flow for this job? Would you say that it ….

□ Has few (if any) slack periods

□ Has few (if any) rush periods (or large build-ups of work)

□ Varies, with fluctuations of rush periods (or large build-ups of work) and slow periods on a daily or weekly basis

3. Can workers step away from their work area easily outside of scheduled breaks (for example, for short bathroom breaks)

□ Yes

□ No

4. How frequently is work checked in the occupation?

□ More than once per day

□ Once per day

□ At least once per week, but not every day

□ Less than weekly

5. Are supervisors generally present in the same work area as workers?

□ Yes

□ No

6. The next question is about “problem solving” tasks that the worker does in his/her job. Think of “problem solving” as what happens when workers are faced with a new or difficult situation which requires them to think for a while about what to do next. How often is the worker responsible for solving problems that take more than 5 minutes to find a good solution?

□ More than once per day

□ Once per day

□ At least once per week, but not every day

□ Less than weekly, but at least once per month

□ Less than monthly, including never

7. How often does this occupation require verbal, work-related interactions?

□ Constantly, every few minutes

□ More than once per hour, but not constantly

□ More than once per day, but not more than once per hour

□ No more than once per day, including never

8. What level of “people skills” does this job require?

□ Minimal/Basic

□ More than basic

Are workers in this occupation:
Required to work with the general public?	Yes No
Required to work around crowds?	Yes No
Required to supervise others as part of job?	Yes No
Permitted to work from home or telework?	Yes No
Required to work outside?	Yes No

• Those are all the questions. Now that you've had a chance to answer each one, I am going to go through them one more time to ask you a few additional questions to help me better understand how you interpreted the question. Here is the first question once again (show card with responses).

1. Which of the following most determines the pace of work on this job? (Select one. Show card with response options)

□ Machinery, equipment, or software controls pace

□ Numerical performance targets or quotas enforced by company determine pace

□ Other persons determine pace (e.g., co-workers, supervisors, customers, members of the public)

□ Other external source (specify) ___________________

□ Self-paced by worker within general performance guidelines

• Did you have any difficulty picking one factor that most controlled the pace of work?

  • Yes – if yes, please explain

  • No

• What does the phrase “Self-paced by worker within general performance guidelines” mean to you?

• Here’s the next question.

2. How would you describe the work flow for this job? Would you say that it ...

□ Has few, if any, slack periods

□ Has few, if any, rush periods (or large build-ups of work)

□ Varies, with fluctuations of rush periods (or large build-ups of work) and slow periods on a daily or weekly basis

• Can you explain what the question means by “work flow?”

• Here’s the next question.

3. Can workers step away from their work area easily outside of scheduled breaks (for example, for short bathroom breaks)

□ Yes

□ No

• Does the [job supervised] have scheduled work breaks? (If yes) How often are these? So, in addition to these regular breaks, it’s not a problem if someone has to take a short bathroom break?

• Here’s the next question.

4. How frequently is work checked in the occupation?

□ More than once per day

□ Once per day

□ At least once per week, but not every day

□ Less than weekly

• What is involved in how you check work in [job supervised)?

• Here’s the next question.

5. Are supervisors generally present in the same work area as workers?

□ Yes

□ No

• In your own words, what does it mean for a supervisor to be “in the same work area as workers?”

• Does a supervisor have to be visible to the worker to be considered to be in the same area?

  • Yes

  • No

  • Depends

6. The next question is about “problem solving” tasks that the worker does in his/her job. Think of “problem solving” as what happens when workers are faced with a new or difficult situation which requires them to think for a while about what to do next. How often is the worker responsible for solving problems that take more than 5 minutes to find a good solution?

□ More than once per day

□ Once per day

□ At least once per week, but not every day

□ Less than weekly, but at least once per month

□ Less than monthly, including never

• This question asked about problem solving in [job supervised]. Can you give me some examples of problems this worker would have to solve?

• Was the 5 minute guideline helpful?

7. How often does this occupation require verbal, work-related interactions?

□ Constantly, every few minutes

□ More than once per hour, but not constantly

□ More than once per day, but not more than once per hour

□ No more than once per day, including never

• Can you think of a different way to ask this question?

8. What level of “people skills” does this job require?

□ Minimal/Basic

□ More than basic

• How would you define minimal or basic people skills?

• Finally, I asked you a series of yes/no questions about the job.

Are workers in this occupation:
Required to work with the general public?	Yes No
Required to work around crowds?	Yes No
Required to supervise others as part of job?	Yes No
Permitted to work from home or telework?	Yes No
Required to work outside?	Yes No

• Who do you consider the “general public” to be?

• (If yes to “work around crowds”) Can you describe the types of crowds this person works around?

• (If yes to “supervise others”) Please describe the types of supervisory responsibilities this work has.

• (If yes to “work outside”) What does it mean to work outside in this job

Attachment C – Consent form

OMB Control Number: 1220-0141

Expiration Date: April 30, 2018

The Bureau of Labor Statistics (BLS) is conducting research to increase the quality of BLS surveys. This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.

The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

During this research you may be audio and/or videotaped, or you may be observed. If you do not wish to be taped, you still may participate in this research.

We estimate it will take you an average of 40 minutes to participate in this research.

Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. OMB control number is 1220-0141 and expires April 30, 2018.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to participate in this study.

___________________________________ ___________________________

Participant's signature Date

___________________________________

Participant's printed name

___________________________________

Researcher's signature

PRIVACY ACT STATEMENT

In accordance with the Privacy Act of 1974 (DOL/BLS – 14 BLS Behavioral Science Research Laboratory Project Files (81 FR 47418)), as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. Per the Federal Cybersecurity Enhancement Act of 2015, Federal information systems are protected from malicious activities through cybersecurity screening of transmitted data.

9