Note to Reviewer - JOLTS Phase 1

December 19, 2014

NOTE TO THE REVIEWER OF:	OMB CLEARANCE 1220-0141
FROM:	Morgan Earp Jennifer Edgar Office of Survey Methods Research

SUBJECT: Submission of materials for cognitive pre-testing of a survey on record-keeping practices in the Job Openings and Labor Turnover Survey: Phase 1

Please accept the enclosed materials for approval under the OMB clearance package number 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the research, and the materials to be used in the research.

On behalf of the BLS, Westat will be conducting in-person and phone interviews with establishment respondents who completed the 2014 JOLTS. These interviews are being undertaken to understand whether the reporting time period for job openings and labor turnover could be changed to match other labor force statistics, like the unemployment rate.

We plan to interview participants from 25 establishments. The BLS estimates that the total burden will be 41.67 hours.

If there are any questions regarding this project, please contact Morgan Earp at (202) 691-7387 or Jennifer Edgar at (202) 691-7528.

Cognitive Pre-Testing to Explore the Reporting of Job Openings in the Job Openings and Labor Turnover Survey (JOLTS)

I. Background and Purpose

The Job Openings and Labor Turnover Survey (JOLTS) is an establishment survey conducted by the Bureau of Labor Statistics (BLS) to generate statistics for employment, job openings, and turnover for key industrial sectors of the United States. The goal of JOLTS is to support BLS’s reporting of unmet labor demand (in the form of rates and numbers of job openings) and turnover. Turnover includes hires and three specific categories of voluntary and involuntary separations (quits, layoffs and discharges, and other separations).

In order for the labor demand measure produced by JOLTS to be comparable to the existing measure of labor supply (the unemployment rate), BLS has worked wherever possible to keep JOLTS consistent with the Current Employment Statistics (CES) program and the Current Population Survey (CPS), a household survey that provides the U.S. unemployment rate.

The goal of this study is to determine the feasibility of aligning the JOLTS data collection and reporting schedule with that of the CES. BLS would like to release JOLTS statistics for each month on the first Friday of the following month, with its other employment indicators. The CES is designed to measure month-to-month changes in employment and is considered to be a more reliable source about trends. Currently, JOLTS data can differ from CES data – due to the differences between the two surveys in reference periods and definitions for measures. Specifically, CES measures employment based on the pay period including the 12th of any given month, and does not measure job openings and hires and separations. JOLTS measures hires and separations for the whole calendar month. JOLTS data for hires and separations are suspected to be undercounts, when compared with data from the Current Population Survey (CPS), a household survey.

The objective of this study is to evaluate the feasibility of aligning the reference periods for the JOLTS data elements with those of the CES. The focus of the qualitative research is to conduct two phases of pretesting activities to evaluate whether JOLTS participants can report JOLTS data for an alternative reporting period. This clearance covers the first phase, for which Westat will conduct 25 in-person interviews for this study. A second phase, to be submitted separately, will include 100 telephone interviews. Specifically, the goals of this study are:

1. Gain insight how JOLTS respondents currently report data, focusing on the current reference period

2. Explore the feasibility of asking JOLTS respondents to report Job Openings for alternate reference periods

   1. the 12^th of the month

   2. the last day of the pay period that includes the 12^th of the month

3. Explore the feasibility of asking JOLTS respondents to report hires and separations for the time between the first day of the pay period following the previous 12^th of the month through the last day of the pay period that includes the 12^th of the current month

   1. Explore options for how to explain this reference period with respondents

4. Identify how soon after the reference period JOLTS respondents would have data available

II. Methodology

A Westat methodologist, accompanied by a BLS staff person when possible, will conduct the interviews at the establishment. The interviewer will follow the guide in Attachment 4. The goals for Phase 1 is to assess participants’ reactions to and feedback about the following:

• Changes to the form related to the reporting period;

• Ease/difficulty of data retrieval and reporting for the reporting periods;

• Data elements that the respondent says are not available for the redefined reporting period and why; and

• Burden in preparing the response for the redefined reporting period.

III. Participants

Twenty-five current and previous JOLTS participants in the Washington D.C. area will be selected for the study. The sample will be distributed evenly across NAICS code. Participants will be recruited via mail (see Attachment 1) and by telephone (see Attachment 2).

IV. Burden Hours

We expect that the in-person interviews will take approximately 60 minutes. We expect recruitment to take approximately 10 minutes, with up 100 establishments contacted to get 25 participants.

Total burden is expected to be 41.67 hours

Phase 1
Total Recruitment Minutes (10 minutes x 100 contacts)	1,000 minutes
Total Interviewing Minutes (60 minutes x 25 interviews)	1,500
Total burden hours	41.67

V. Payment

Participants will not receive payment for this study.

VI. Confidentiality

For face-to-face interviews, participants will be asked to sign a consent form (Attachment 3).

For telephone interviews, the following Privacy Act statement and confidentiality assurance will be relayed to the respondent during recruitment as well as in the introduction script at the beginning of the interview. Participants will be informed the interview is voluntary and the information collected will be used for internal research purposes only.

“Your voluntary participation is requested to help BLS understand business records and their use in designing a more effective long-term approach to collecting and publishing these new data types. The purpose of this call is for internal purposes only. Information related to this study is confidential and will not be released to the public in any way that would allow identification of individuals or establishments. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.”

V. Attachments

1. Recruitment Letter

2. Recruitment Script

3. Consent Form

4. Phase 1 Interviewer Guide

5. Current JOLTS Form

6. Alternate JOLTS Form

Attachment 1: Recruitment Letter

U.S. Department of Labor

Bureau of Labor Statistics 2 Massachusetts Avenue, NE, Room 1950 Washington, DC 20212

Thank you for your recent participation in the Job Openings and Labor Turnover Survey (JOLTS) conducted by the U.S Bureau of Labor Statistics (BLS). Your participation in this important survey has allowed us to provide policy makers, researchers, and the general public with valuable information on current developments in the labor market, especially in the areas of labor demand and employee turnover. Without the cooperation of your firm and others, we would not be able to generate current and accurate information about these critical labor market measures.

To more fully understand the forces impacting the current labor market, the Bureau of Labor Statistics is conducting research on the possibility of accelerating our collection period to make our estimates of openings, hires, and separations even more timely and useful.

The Bureau of Labor Statistics is conducting a study to understand whether the reporting time period for job openings and labor turnover could be changed to match other labor force statistics, like the unemployment rate. We routinely ask businesses to help us during the survey design process to assure that we collect the best survey data possible.

To make sure we collect accurate data, we need to better understand:

• The types of information you use when reporting job openings and labor turnover;

• Whether survey form directions and definitions are clear; and,

• Whether changes in the reporting time period make it easier or more difficult to provide the requested data.

The research involves a Westat researcher visiting you at your establishment, completing a JOLTS form, and then discussing your feedback on the form. Westat, the contractor for the study, will contact you by phone in the coming week to request an appointment with you during the period from [INSERT START DATE] through [END DATE]. The interview will take no more than 60 minutes of your time.

BLS would appreciate your cooperation with Westat. This is an important study that will provide useful information to help develop better reporting methods for job openings and labor turnover in the United States. If you would like additional information please contact the Westat Project Director, Gina Shkodriani at 240-453-2904.

Sincerely,

Name

Title

Bureau of Labor Statistics

U.S. Department of Labor

Attachment 2: Recruitment Script

Job Openings and Labor Turnover Survey (JOLTS)

Accelerated Collection Testing

Recruitment Script

Hello, my name is ___________________ and I’m calling on behalf of the Bureau of Labor Statistics (BLS) of the U.S. Department of Labor about a study on the Job Openings and Labor Turnover Survey, or JOLTS. I understand that _____________________ (you/name of contact person) are/is currently the contact person for completing the monthly JOLTS form for the Bureau of Labor Statistics.

IF THIS IS NOT THE CORRECT PERSON: Who would be most knowledgeable about completing the JOLTS form? _______________________ May I please speak to him/her?

Your voluntary participation is requested to help BLS understand business records and their use in designing a more effective long-term approach to collecting and publishing these new data types. The purpose of this call is for internal purposes only. Information related to this study is confidential and will not be released to the public in any way that would allow identification of individuals or establishments. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.”

BLS is interested in finding out how easy or difficult it would be for establishments like yours to use a different reporting period than you currently use for completing the JOLTS report. BLS randomly selected your establishment from their database of establishments currently providing employment data on the JOLTS forms. We would greatly appreciate it if you would participate in the study. If you decide to participate, one of our researchers will visit your establishment for no more than one hour. During the visit, we’d like to learn about how you currently complete the JOLTS form, and get your feedback about some proposed changes to the form. We will be interested in learning your thoughts and opinions about the data collection process.

Would you be willing to schedule a time for us to visit sometime from [INSERT START DATE] through [END DATE]?

□ Yes □ No __________ Date __________Time

Establishment Name _______________________________________

Contact Person ________________________________________

Phone number ________________________________________

Email address ________________________________________

Address ________________________________________

Respondent’s Name ________________________________________

Respondent’s Title ________________________________________

Respondent’s Phone ________________________________________

Respondent’s Email address_____________________________________

Attachment 3: Consent Form

U.S. Department of Labor

Bureau of Labor Statistics 2 Massachusetts Avenue, NE, Room 1950 Washington, DC 20212

You will be taking part in a study to test alternative ways of reporting data for the Job Openings and Labor Turnover Survey (JOLTS). This study is being conducted by Westat for the Bureau of Labor Statistics (BLS). During this interview, you will answer a series of questions about how you complete the JOLTS form, and about your reactions to proposed changes to some of the instructions on the form. There are no right or wrong answers to the questions we will ask you.

The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.

During this research you may be audio and/or videotaped, or you may be observed. If you do not wish to be taped, you still may participate in this research. We estimate it will take you an average of 45 minutes to participate in this research (ranging from 30 minutes to 60 minutes). Your participation in this research project is voluntary, and you have the right to stop at any time.

If you have questions about the study, please contact the Westat Project Director, Gina Shkodriani at 240-453-2904.If you have questions about your rights and welfare as a research participant, please call the Westat Human Subjects Protections office at 1-888-920-7631. Please leave a message with your full name, the name of the research study that you are calling about (Job Openings and Labor Turnover Survey (JOLTS)), and a phone number beginning with the area code. Someone will return your call as soon as possible.

Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. OMB control number is 1220-0141 and expires 02-28-2015.

If you agree to participate, please sign below.

------------------------------------------------------------------------------------------------------------

I have read and understand the statements above. I consent to participate in this study.

___________________________________ ___________________________

Participant's signature Date

___________________________________

Participant's printed name

___________________________________

Researcher's signature

OMB Control Number: 1220-0141

Expiration Date: 02-28-2015

Attachment 4: Phase 1 Interview Guide

Testing Alternative Reporting Periods for the

Job & Labor Turnover Survey (JOLTS)

ROUND 1 INTERVIEW GUIDE

Session Materials:

• Interview Guide

• Consent form

• Blank Calendar

• Current JOLTS form (if needed for Respondent to reference)

• Alternate JOLTS revised form (if time allows for review)

• Green and Red Pens

RESPONDENT VERIFICATION

• Confirm that Respondent is the person we are scheduled to meet with

INTRODUCTION

Hello, my name is ______________ and I work for Westat, a survey research company in Rockville, Maryland. Thank you for taking the time to meet with me today. We know you are busy and we appreciate your time and willingness to help. We expect that our interview with you today will take one hour.

Westat is working with the Bureau of Labor Statistics (BLS) to study the Job Openings and Labor Turnover Survey (JOLTS), an establishment survey conducted by BLS to generate statistics for job openings and turnover for key industrial sectors of the United States. The goal of JOLTS is to support BLS’s reporting of unmet labor demand (in the form of rates and numbers of job openings) and turnover.

BLS has come up with some changes to how employers report their JOLTS data to allow it to be more comparable to measures of labor supply, such as the unemployment rate.

We are talking to companies that currently fill out the JOLTS form to collect feedback about the changes that could be made to the reporting form. Before changes such as this are made to surveys, it’s important to test them out with the help of people such as yourself to make sure that the instructions make sense, are easy to follow, and that everyone understands them the same way. Outcomes from our discussion will enable us to learn the feasibility of changing the way we collect data. There are no right or wrong answers, we’re looking for your reactions and honest feedback.

CONFIDENTIALITY and INFORMED CONSENT

Before we get started, there are a few things I want to mention. Your participation is voluntary. If you prefer not to answer a question, just say so and we’ll go on to the next one. You have the right to stop the interview at any time.

All your answers, everything you say, will be kept confidential. Neither you, nor your company will be identified in any reports. (For telephone Interviews read the following: The purpose of this call is for internal purposes only. Information related to this study is confidential and will not be released to the public in any way that would allow identification of individuals or establishments. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.” The survey is being collected under OMB Control Number 1220-0141.)

The interview will take about an hour. I would like to audio record our conversation. This helps me so I can listen to what you are saying and won’t have to take a lot of detailed notes while you are talking. It will also help when I write up a summary of this interview. Only project staff will have access to the recording and other project materials and those materials will be stored in a secure location at Westat.

[IF OBSERVERS: Finally, some of the researchers involved in revising the form are here today observing our interview to learn if there are things that might need to be changed.]

(For In person interviews) HAND CONSENT FORMS TO RESPONDENT: This form contains all of the things I just told you about your rights in this interview. Please read it over and sign one copy if you are willing to take part in the study. You can keep the other copy for your records.

R SIGNS ONE CONSENT FORM FOR PROJECT & KEEPS OTHER COPY FOR SELF.

TURN ON RECORDER. The date and time is ____________. Now that the recorder is running, let me ask again; is it okay with you if we record this interview?

Thank you. I’d like to start by learning a little about this establishment and your role here.

1. RESPONDENT INFORMATION

A1. What division or department do you work in here?

A2. What is the title of your position here?

A3. How long have you worked for [ESTABLISHMENT]?

A4. Tell me, briefly, what you do here.

A5. Are you the one who usually fills out the JOLTS form for your company?

A6. Up to today, have you needed to seek input/data from others in your firm (e.g. payroll, human resources or other departments) to complete the JOLTS form?

A6a. If Yes, who and why?

2. EMPLOYER INDUSTRY & PAYROLL DETAILS

B1. How often are employees paid here? (Note: It is OK to select more than one)

Each week	Once a month
Every two weeks	Other Please describe:________________________
Twice a month

B1a. Is there anything about the frequency of your payroll that has made it difficult in the past to complete the JOLTS form?

B1b. If Yes, can you tell me what is difficult?

B2. According to BLS records, your establishment is in the [NAICS DESCRIPTION OF INDUSTRY] industry. Is this correct?

• YES (GO TO SECTION C.)

• NO (CONTINUE)

B3. [IF NO] I’m going to read a list of industry groupings. Please tell me which one best describes your establishment’s industry:

Agriculture, Forestry, Fishing and Hunting	Information
Mining, Quarrying and Oil and Gas Extraction	Finance and Insurance
Construction	Real Estate, Rental and Leasing
Manufacturing	Professional and Business Services
Wholesale Trade	Education and Health Services
Retail Trade	Leisure and Hospitality
Transportation and Warehousing	Public Administration
Utilities	Other Services (except Public Administration)
B3a. RECORD INDUSTRY HERE:

3. Employment

For this study, we’re interested in how you understand and report data for specific reference month. For this interview, we’ll be talking about [fill month]’s data.

Let’s start with Employment. We currently ask for total employment for the pay period that includes the 12^th of the month. Take a look at the calendar for [FILL: last month].

C1. What pay period included the 12^th of the [fill month]?

C1a. What was the start date of that pay period?

C1b. What was the end date of that pay period?

C2. After that pay period is over, how quickly do you have the necessary information (records, listings, reports, etc.) to report total employment?

4. Job Openings

The current JOLTS form asks you to provide “the number of job openings on the last business day of [fill month].”

D1. Looking at the calendar, what was the last business day of the month for your establishment?

D1a. Are you able to provide the data for the last day of the month or do you use another date or time period?

D1b. How did you identify which data to provide for the last business day of the month? (E.g. Where do you get the data about the number of job openings on the last business day of the month?)

D1c. After that pay period is over, how quickly do you have the necessary information (records, listings, reports, etc.) to report number of job openings?

D2. For job openings, we are thinking about asking for the total number of job openings that you had on the 12^th of the current month. Would you be able to report that information?

D2a. When would the information you would need about job openings on the 12^th of the current month be available? (e.g. how long does it take for the necessary records, listings, reports, etc., to be processed and become available?)

D3. We currently ask for Job Openings data for the last business day of the month. Would the process for providing it for the 12^th of the month be any different? How so?

D3a. Do you think it would be easier, harder or about the same to provide job openings data for the 12^th of the month rather than the last business day of the month? Probe: Why would it be easier/harder?

D4. Another option we’re considering is to ask for Job Openings data for the last day of the pay period that includes the 12^th of the month. Would you be able to report that information?

D4a. When would the information you would need about job openings on the last day of the pay period that includes the 12^th of the month? (e.g. how long does it take for the necessary records, listings, reports, etc., to be processed and become available?)

D4b. Do you think it would be easier, harder or about the same to provide job openings data for the last day of the pay period that includes the 12^th of the month? As you mentioned earlier, that would be (Fill C1b). Probe: Why would it be easier/harder?

D5. Would it be easier to report Job Openings for the last day of the pay period that includes the 12^th of the month or for the 12^th of the month? Why?

D5a. Thinking about the two options, reporting job openings for the 12^th of the month or for the last day of the pay period that includes the 12^th of the month, would you be able to report one earlier than the other? Why?

5. Hires

E1. We currently ask for the number of hires for “the entire month.”

E1a. Are you able to provide the data for the entire month or do you use another date or time period?

E1b. How do you currently get the answer to this question? (E.g. Where do you get the data about the number of hires for the entire month?)

E1c. After that pay period is over, how quickly do you have the necessary information (records, listings, reports, etc.) to report number of hires?

E2. We are thinking about changing this question to instead ask for hires for a different period. Rather than asking about a full calendar month, it would be a reference period from the middle of one month to the middle of the next month. Would moving away from the calendar month be difficult for you?

E3. We’re trying to figure out the best way to help respondents understand the new reference period. We have drafted some steps we’d like you to try and give your feedback on. To find the start date of the reference period,

E3a. Identify the last business day of the pay period that includes the 12^th of last month.

E3b.Then find the first business day of the following pay period.

E3c. That’s (first business day of the following pay period) the start date of the proposed new reference period. Please circle it in green on the calendar.

E4. To find the end date of the new reference period

E4a. Identify the pay period that includes the 12^th of the current month.

E4b. Then find the date of the last business day of that pay period.

E4c. That is the end date of the new period. Please circle it in red.

E5. What do you think about those steps?

E5a. We realize it’s not a simple process. How would you explain it to a colleague?

E5aE5b. Is there an easier way to explain the new reference period?

E5c. Can you think of a term or words that would best describe the new reference/time period?

E6. Now, looking at that new reporting period you circled, would you be able to report hires for that time period? (e. When would you have that information available?)

6. Separations

F1. We currently ask for the number of separations for “the entire month.”

F1a. Are you able to provide the data for the entire month or do you use another date or time period?

F1b. How do you currently get the answer to this question? (E.g. Where do you get the data about the number of separations for the entire month?)

F1c. After that pay period is over, how quickly do you have the necessary information (records, listings, reports, etc.) to report number of separations?

F2. Just like hires we are thinking about changing this question to instead ask for separations for a different period. Rather than asking about a full calendar month, it would be a reference period from the middle of one month to the middle of the next month. Would moving away from the calendar month be difficult for you?

F2a. We’d also be looking for information about separations for that new reference period, the one we just circled on the calendar. Would you be able to report for that time period?

F2b. When would the information you would need about separations for this new reference period be available? (e.g. how long does it take for the necessary records, listings, reports, etc., to be processed and become available?)

F2c. Do you think it would be easier, harder or about the same to provide separations for this new reference period than for what you current report, ‘the entire month’?

7. NEW REFERENCE PERIOD: General Feedback

G1. What, if any, any concerns would you have about BLS changing the reference period that you use to report for the Job Openings and Hires questions?

G2. Would you have any problems changing the dates for which you report information to JOLTS?

G2a. If so, please briefly describe what those would be and why.

8. ALTERNATE FORM, V1 {if session time permits}

As I mentioned, we have a draft form that includes these new reference periods, and I’d like to get your feedback on it. [INTERVIEWER: HAND RESPONDENT REVISED JOLTS FORM]

Up to this point we’ve talked through some of the main changes that relate to changing the reference period for some of the questions.

H1. Take a look at the box marked 3 on the revised version of the form. Please follow the instructions and fill in the blank date fields.

H1a. Please walk me through what you were thinking about when you came up with the dates.

H1c. How easy or difficult was it for you to come up with these Start and End Dates? Probe: What made it easy/difficult?

H1d. What, if anything, is unclear about these instructions? Probe: What might help respondents understand the time period we’re looking for?

H2. Now let’s look at the box marked 4. I’m going to have you show me on the calendar the time period that each column is asking you to provide data for. Using the calendar…

H2a. Show me the time period that you would use to report Total Employment in Column A.

H2b. Show me the time period that you would use to report Job Openings in Column B.

H2c. Show me the time period that you would use to report Hires in Column C and Separations in Columns D through F.

9. WRAP UP AND CLOSING

Those are all of the questions that I have for you today.

I1. Is there anything that we have not already discussed that you would like BLS know about using the alternative reference period to complete the JOLTS form?

Thank you for your time.

Attachment 5: Current JOLTS Form

Attachment 6: Round One Test JOLTS Form