Attachment B Debriefing Protocol

Attachment B. Employee Interview Protocol

Introduction

Thanks for agreeing to participate in our study. We are looking for information about how companies train their employees and what information they keep about training. We’ll ask you a series of questions, please answer to the best of your ability – but if we ask something you do not know the answer to, please tell us that as it’s likely other people wouldn’t know either.

The information you provide will be used for research purposes only, and only seen by the researchers involved in the project. I expect the interview to take about an hour, and really appreciate your time. Any questions before we begin?

Respondents will read and sign consent form.

General Survey Feedback

1. Thanks for completing the survey. Overall, how was it for you?

2. How long did it take you to fill out?

3. Did any of the questions stand out to you for any reason?

4. Were any of the questions particularly hard? Were you unable to answer any?

5. Would you have any concerns filling this survey out again?

Employee Information Section

Now I’d like to go through some of the specific questions on the survey to get your reactions. Starting with the first page, where we asked about you and your job.

1. Looking at this first section of the survey, were any of these questions difficult?

2. We asked you to categorize your job. Can you tell me how you selected [fill response]?

   1. Did the categories make sense to you?

3. Did the answer choices in number 7 make sense? Were you sure how to categorize your company?

4. What about number 10, how confident are you that your answer is right?

5. What is the name of the company you work for? What address is your work location?

Formal Training Section

The next section asked about formal training offered by your company.

1. Did the definition of formal training make sense to you?

   1. Is formal training something you’ve heard of before?

   2. Were there any other types of training that you wondered if should be included?

   3. Does your company offer any other types of training that you think should be listed?

2. We asked you to think about formal training over the last 12 months. Was that easy or difficult?

   1. Tell me about the process you used to remember the training you received

   2. Do you think that you were able to accurately remember the training you got 12 months ago?

   3. What about 6 months ago? 3 months?

   4. Our goal is to get as much information as possible, so we’d like to ask about a year, but know that it may be difficult for people to remember that far back. How far back do you think we should ask people about?

3. We gave you a table and asked you to mark off training you received, broken out by topic and type of training. Let’s look at the columns first. Did each of these categories make sense to you?

   1. Were any less clear than others?

   2. Does your company offer each of these to its employees?

4. Can you give me examples of:

   1. Traditional, in-person, classroom training

   2. Other in-person training

   3. Online training

   4. Combination of in-person and online training

   5. Some other type of formal training

5. Now, looking at the training topics. Did each of these categories make sense to you?

   1. Were any less clear than others?

   2. Does your company offer each of these to its employees?

6. You noted that you’d received [fill cells marked yes]. Can you tell me about that training?

   1. [probe for mode/topic/source/timeframe]

   2. How sure were you that you put it in the right box?

7. In question 13, we asked about where your training came from. Did this question make sense to you?

   1. Are you familiar with the source of the training you take?

   2. You said your training came from [fill answers], how do you know that?

   3. [probe for any inconsistencies from grid]

8. The last question in this section asked if your employer sponsored or financed you taking classes for a degree related to your current job. What does that question mean to you?

   1. You said [fill answer]. Can you tell me more about that?

   2. Who should say yes to that question?

Informal Training Section

The next section of the survey asked about informal training.

1. Did the definition of in formal training make sense to you?

   1. Is informal training something you’ve heard of before?

   2. Were there any other types of training that you wondered if should be included?

   3. Does your company offer any other types of training that you think should be listed?

2. Like the previous section, we asked you to indicate the types of informal training you’ve received from your company in the last 1 month.

   1. For this type of training, we thought it might be more difficult for people to remember, so are only asking about one month’s worth. Do you think you were accurately able to remember all the informal training you’ve received in the past month?

   2. Would you be able to answer the questions accurately for the last 3 months?

   3. How far back do you think people could accurately remember this information?

3. Looking at the columns, we included several types of informal training. Did each of these make sense to you?

4. Can you provide examples of:

   1. Mentoring

   2. Job shadowing or demonstrations

   3. Learned on my own at work

   4. Other informal training

5. Here we used the same training topics. Do they all make sense to you when thinking about informal training?

   1. Are any confusing or difficult in this context?

6. You said [probe for each checked box]. Can you tell me about that?

Employee Demographics Section

The last section of the survey asked for some background information about you.

1. What is your reaction to this section?

2. Why do you think we’d ask this information on a survey about training?

3. Did any of the questions stand out to you for any reason?

4. Were any difficult to answer?

   1. Confusing?

5. Question 18 asked for your pay. How did you go about figuring out your answer?

6. Question 19 asked how many weeks you work in a year. How did you go about figuring out your answer?

Closing

Thank you for your feedback on this survey, it’s been very helpful! Do you have any other comments or suggestions before we wrap up?

Attachment C. Employer Interview Protocol

Introduction

Thanks for agreeing to participate in our study. We are looking for information about how companies train their employees and what information they keep about training. We’ll ask you a series of questions, please answer to the best of your ability – but if we ask something you do not know the answer to, please tell us that as its likely other people wouldn’t know either.

The information you provide will be used for research purposes only, and only seen by the researchers involved in the project. I expect the interview to take about an hour, and really appreciate your time. Any questions before we begin? [Administer consent form/ read: The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141. We will use the information you provide for statistical purposes only. Your participation is voluntary, and you have the right to stop at any time. We estimate it will take 60 minutes to complete this research study.

Experience Completing Survey

1. Thanks for completing the survey. Overall, how was it for you?

2. How long did it take you to fill out?

3. Did any of the questions stand out to you for any reason?

4. Were any of the questions particularly hard? Were you unable to answer any?

5. Would you have any concerns filling this survey out again?

Company Information

The first set of questions asked for background information about your company.

1. Were any of these questions difficult to answer?

   1. Confusing?

2. Question 3 asked for the number of employees on the payroll on the 12^th of last month. How did you go about answering that question?

   1. Did you use the same process when asked about three months ago in question 4?

3. Question 5 asked what type of staff work at your company. Were you familiar with all of the types of workers?

   1. [if answered on call] You reported having on-call workers, can you tell me about those?

   2. [if answered temp help agency] You reported having temporary help agency workers, can you tell me about those?

   3. [if answered independent contractors] You reported having independent contractors, can you tell me about those?

   4. [if answered contract company workers] You reported having contract company workers, can you tell me about those?

   5. [if answered other] You said you also had [fill response]. Can you tell me about those?

4. Question 7 asked “Which of the following services does your company contract for?” What does that question mean to you?

5. Same for Question 8, what does this question mean to you “Does your company supply contract workers or temporary workers to other companies?”

Formal Training

1. One set of questions asked about formal training. Can you give me some examples of formal training?

2. Was it clear what training should be include as formal training? Excluded?

   1. Was there any training that your company offered in the last X months that you weren’t sure whether to include?

3. [Q10] You said that your company has / has not provided or financed any formal training for employees. Can you tell me about that?

4. [show Q11 grid] This is one of the questions you answered. Did all the training topics make sense to you? Are there any that you aren’t familiar with?

   1. [if other marked] tell me about that training.

5. Now look at the column titles, did all of these types of training make sense?

   1. Does your company offer each of these?

   2. Are there other types of training your company offers that isn’t listed here?

   3. [if other marked] tell me about that training.

6. [probe to find out about the individual trainings indicated in the grid. Get enough information to ensure that the right box is marked]

Formal Training Section

The next section of the survey asked about formal training.

1. Did the definition of formal training make sense to you?

   1. Is formal training something you’ve heard of before?

   2. Were there any other types of training that you wondered if should be included?

   3. Does your company offer any other types of training that you think should be listed?

2. We asked you to think about formal training over the last 12 months. Was that easy or difficult?

   1. Tell me about the process you used to determine if formal training had been sponsored or financed in the last 12 months.

   2. Do you think that you were able to accurately remember the training your company sponsored or financed in the last 12 months?

   3. What about 6 months ago? 3 months?

   4. Our goal is to get as much information as possible, so we’d like to ask about a year, but know that it may be difficult for people to remember that far back. How far back do you think we should ask people about?

3. We gave you a table and asked you to mark off training you received, broken out by topic and type of training. Let’s look at the columns first. Did each of these categories make sense to you?

   1. Were any less clear than others?

   2. Does your company offer each of these to its employees?

4. Can you give me examples of:

   1. Traditional, in-person, classroom training

   2. Other in-person training

   3. Online training

   4. Combination of in-person and online training

   5. Some other type of formal training

5. Now, looking at the training topics. Did each of these categories make sense to you?

   1. Were any less clear than others?

   2. Does your company offer each of these to its employees?

6. You noted that your company had sponsored [fill cells marked yes]. Can you tell me about that training?

   1. [probe for mode/topic/source/timeframe]

   2. How sure were you that you put it in the right box?

7. In question 13, we asked about where your training came from. Did this question make sense to you?

   1. Are you familiar with the source of the formal training offered?

   2. You said your training came from [fill answers], how do you know that?

   3. [probe for any inconsistencies from grid]

Employees in Formal Training

In the next section, we asked questions about employees that took formal training from your company.

1. What is your reaction to these questions?

2. Were any of them difficult to answer?

3. How confident do you feel in the accuracy of your answers?

4. Question 14 asked about the percent of employees who received training in the last 12 months. How did you go about answering that question?

   1. Do you think your answer is accurate?

   2. Again, we’re trying to balance the amount of information with the difficulty of the question. Do you think 12 months is a reasonable time frame for this question?

   3. How long back do you think you could answer for and be completely confident in your answer? A month? 3 months? 6 months?

5. In Question 15, we asked the same question but for you to exclude new hires and orientation. What was your process for answering that question?

   1. [if necessary] Did it differ from answering about all formal training?

   2. Do you think your answer is accurate?

   3. Would you recommend the same or different time period for this question?

6. We also asked the number of hours each employee has spent, on average, in formal training over the last 3 months. Please tell me how you went about answering that question.

   1. Was this question easy or difficult to answer?

   2. Do you think your answer is correct? How off might it be?

   3. What time period could we use to ensure that you were able to get an accurate response?

Spending on Formal Training

1. The last section asked if your company has spent money on training in the last 12 months. What is your reaction to that request?

   1. Did each of the categories make sense?

   2. Can you explain what each means:

      1. Tuition

      2. Outside trainers or training companies

      3. Wages and salaries of full time training personnel

      4. Wages and salaries of part time training personnel

      5. Other training related expenses

         1. [if selected] What did you include here?

   3. If we had asked you to report how much money the company has spent on each of these types of training, would you have been able to answer that question?

      1. For the last 12 months? 6 months? 3 months?

      2. How would you get that information?

Employee Background

Before we wrap up, I’d like to get some information about your role in your company. This will help us understand your situation and how it may be similar or different from others.

1. What is your job title?

2. Can you tell me a little about your role in the company?

3. What department do you work for?

4. How long have you been with this company?

5. How long have you been doing this type of work?

6. Do you think you would be the one who would receive this type of survey?

   1. Would you answer it?

Closing

Thank you for your feedback on this survey, it’s been very helpful! Do you have any other comments or suggestions before we wrap up?