Attachment C Debriefing protocol

OMB Control Number: 1220-0141

Expiration Date: 03/31/2021

Attachment C – Interview protocol

Introduction script

Hi, may I speak to (respondent name)

My name is ___. I work for the Office of Survey Methods Research at the Department of Labor, Bureau of Labor Statistics.

We are contacting you because you recently participated in a study for the Occupational Employment Statistics (OES) report – thank you! We’re contacting businesses like yours to learn more about how we can make the response process easier and simpler in the future, and we’d like to get your feedback on completing the report. This study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141. This voluntary survey will just take about 10-20 minutes. The information you provide will be kept confidential and will be used for research purposes only.

If you have a few minutes now, we’d like to get your feedback.

(If not, ask to schedule a call-back time)

(If yes) Great, do you have access to your email at the moment? There are some documents that will help to see during our conversation

<Verify email address if there is one on file, and send screen shots of the data collection instruments>

<If no email address is on file, collect email address>

<Verify they received the email>

<Read to respondent>

The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act (44 U.S.C 3572) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. Per the Federal Cybersecurity Enhancement Act of 2015, Federal information systems are protected from malicious activities through cybersecurity screening of transmitted data. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.

General Feedback

The OES report collects occupation and wage information from businesses. Do you recall completing this form in the past 3 months or so?

• Yes

• No (Okay, well hopefully walking through this form together will help with remembering it.)

Great, let’s take a look at that email I sent you, on page two of Form A/B <reference the form the respondent actually completed>. You should see a page that says “Occupational Employment Statistics Form” at the top, with a large table on it. Do you see that?

• Thinking back to when you completed the report, what were your general reactions to this page?

• Looking again at the form and the instructions, what do you think it’s asking you to report?

Data Quality Probes

We’re interested in learning how businesses complete our forms. I’ve pulled up the data you submitted for reference, we just have a few follow-up questions for you.

• In looking at the form you submitted, we noticed [insert probes appropriate to the respondent’s data]:

• The total number of employees at your business did not match the number of employees listed on the form. Can you tell me more about that?

• You indicated that you provided an average wage rate for a group of employees in a particular occupation. Can you tell me more about that?

• The number of unique job titles listed on the form is lower than other establishments of your same size class. Was that correct?

[Other emergent probes as needed, tailored to the respondent’s form]

• Was there anything confusing or unclear about this form?

Column probes

(For those not assigned to column form) Now let’s pull up the second pdf I sent you, labeled Form A. Go to page 2. This form is similar to the one you completed, except now there is a column to indicate the number of employees in each job who earn the same salary. Do you see that column?

• What are your reactions to this column?

• Do you think you would be able to use it?

• Would it have changed how you reported your data? If so, how?

• Would this column have made it easier or more difficult to report your data?

(For those who were assigned to the column form) Now let’s pull up the second pdf I sent you, titled Form B. Go to page 2. This form is similar to the one you completed, except now there is no column provided to indicate the number of employees in each job who earn the same salary. Instead, you would list out each employee individually on separate rows.

• What are your reactions to this?

• Would not having this column have changed how you reported your data? If so, how?

• Would using this form (without the column) have made it easier or more difficult to report your data? How so?

Point Estimate Feedback

Now we have some questions about providing the wage rates for your employees.

• Do employees with the same job title and duties typically make the same wages, or does it vary?

  • How much does it vary?

  • Would you be able to list each occupation that makes a different wage separately? How would you go about that?

• Do you think it would be easy or difficult to provide exact wages for each of your employees?

[Other emergent probes as needed, tailored to the respondent’s form]

[Probes to select from; not all probes will be asked. At researcher’s discretion.]

• What records or reports did you have to access to get this information?

• Do those give you this information directly, or would you have to do something custom or manual?

• What challenges might be involved?

• Would this require additional consultation with other employees?

• Looking up records in another place?

• Generating a spreadsheet from payroll software?

• Recall from memory?

• Calculations?

• Guessing or estimating?

Burden estimate

• How long would you estimate it took you to complete the form?

• Which form would you prefer – Form A (the one with the column for the number of employees), or Form B (listing each employee on separate rows)?

Conclusion

• Do you have any other feedback on the form you completed?

• Do you have any feedback on how we could make it simpler and easier for you to complete the OES report in the future?

• Thank you for talking to us! We appreciate your time.

E-mail text to be sent:

Hello,

Thank you for agreeing to help the Department of Labor, Bureau of Labor Statistics with a brief research study to improve our data.

Please see the attached pdfs titled “Form A” and “Form B” for your reference during our phone call.

Thank you again for your help!

You should be aware that use of email involves certain inherent risks to the confidentiality of your data.  We advise sending this information in an encrypted manner.

<insert forms>

4