Form ASM protocol ASM protocol ASM protocol

Attachment A - 2020 ASM Covid Questions and Revised Employment/Payroll PROTOCOL

Cognitive Interviews

9/2/2020

Research Questions:

• Do respondents understand what information they are being asked to provide with these questions?

• Do respondents understand the terminology used in the questions?

• Are respondents able to answer the questions that are being asked?

• Is the requested information available in respondents’ records?

• Does the revised order of the Payroll and Employment questions make sense to respondents?

• Does the order of the questions about donations (Sales first then Special Inquiry) make sense to respondents?

Expected Length of Interview: 30 minutes

Materials Needed:

• Draft question and ASM form

General probes that may be used throughout the interview:

• Reflect back on respondent’s answer: “you said…”

• In your own words, what is this question asking?

• How did you arrive at this number/answer this question?

• What records (if any) did you look at? What line or lines were of interest?

• Specifically, what did you include in this number? What did you exclude?

• Would you consult other people to obtain this answer?

INTRODUCTION

• Explain purpose of meeting: Thank you very much for your time. Let me start by telling you a little about what we will be doing today. We are asking for feedback new questions that are being added to the Annual Survey of Manfactures (ASM) along with looking at a revision that we have made to an existing questions to ensure that it makes sense. We would like to get your input on the new and revised questions in order to make sure they are as clear and easy to complete as possible. Any information you provide will be kept confidential

ABOUT THE RESPONDENT

First, I’d like to learn a bit about you, and your role here at the company.

• What is your job title?

• What is your role in completing government surveys?

• How much experience do you have completing the ASM?

• What has your experience completing the ASM been like?

• Do you work with anyone else in your company to get the data for the ASM?

  • How are those other people involved?

  • How many people are involved?

Thank you very much for that background information! Now, I would like to show you some new and revised questions for the 2020 ASM.

CLOSED DAYS

How many days during 2020 was your establishment closed due to the coronavirus pandemic?

Q1 (January 1, 2020 – March 31, 2020)

Q2 (April 1, 2020 – June 30, 2020)

Q3 (July 1, 2020 – September 31, 2020)

Q4 (October 1, 2020 – December 31, 2020)

• Can you tell me a little about how you would go about answering this question for any manufacturing establishments that you are responsible for reporting on for the ASM?

• How confident are you in your response?

  • How hard is it for you to come up with your response?

  • Would a range of the number of days make this question easier or more difficult to answer?

    • What range of days makes sense to you here?

• Did any of the establishments in your company experience closures due to the pandemic?

  • If yes, did your company track this information?

• Although we haven’t finished 2020, how would you go about answering this question for the three quarters of the year?

  • Do you have access to this information?

  • If you do not have access to this information, would others in your company have access to this information?

ITEM 5 - DONATIONS

This question currently appears on the form. The instructions for the 2020 have been expanded.

ITEM 5: SALES, SHIPMENTS, RECEIPTS, OR REVENUE

A. What was the total value of products shipped and other receipts for this establishment? (Report detail in item 22)

Include:

o All products physically shipped from this establishment during 2020

o Products donated and physically shipped from this establishment during 2020

Exclude:

o Freight charges

o Excise taxes

• How would you go about answering this question?

• Can you tell me how you would handle the include statement about “products donated and shipped from this establishment?”

  • Did any of the manufacturing establishments in your company donate products in 2020?

  • IF YES,

    • Can you tell me a little about the products that you donated?

    • Is the value of your donations already included within any total Sales, Shipments, Receipts, or Revenue values that you have on your books?

      • If not, how would you go about including the value of donations made in 2020 to your total figure?

A. Did this establishment donate any products during 2020?

Yes

No

B. What was the value of the donated products? (This is a breakout of the $,000 reported in Item 5, line A.)

IF YES TO A

• Are you able to provide the value of your donations for 2020?

• How do you track donations in your records?

• Is this figure easily accessible to you?

• Would you have to work with any others in your company in order to identify the value of donations?

ITEM 7 - PAYROLL AND EMPLOYMENT (excluding D. Hours Worked and E. Fringe Benefits)

Remind the respondent to think aloud as they answer this item. Listen for any pauses or areas of confusion because of the order of the questions.

• Can you tell us how you would go about providing the quarterly payroll figures for production workers?

  • Do you have this information readily available? Would you have to do any manipulations/calculations with your data in order to get these values?

  • Would you have to work with others in your organization to access this information?

• The question asks for overall company totals and then asks about breakdowns by production workers and all other workers. What are your thoughts on the format of this question?

  • Did the order make sense?

  • Would your experience change if the question asked about details first followed by totals?

ITEM 22 - DETAIL OF SALES

This question currently appears on the form. Each manufacturing establishment receives a custom list of products that we believe they produce. For the 2020 survey, we will be adding the following 6 products to ALL forms. Please read through the question or orient yourself with what is asking along with just the 6 new product lines being added.

ITEM 22: DETAIL OF SALES, SHIPMENTS, RECEIPTS, OR REVENUE

Of the $,000.00 of Sales, Shipments, Receipts, or Revenue reported in Item 5, what was the value of each product or service?

General – Please do not combine product lines. If the information is not directly available from your records, reasonable estimates are

acceptable.

The manufactured products and services listed below are generally made in your industry. If you make products or have revenue from sources not listed, click the “Add Product Not Listed” button and search for an existing product, or use the section for “Add product(s) not listed above.”

Manufacturing of Products – Report the value of the products shipped and services performed at the net selling value, free on board (FOB) plant to the customer, after discounts and allowances.

Include:

Products made elsewhere by others from materials supplied by this establishment. Report these products on the specific lines as if they were made in this establishment.

Products transferred to other establishments within your company. These products should be assigned the full economic value (market value); i.e., include all direct costs of production and a reasonable proportion of all other costs and profits.

Exclude:

Wholesale products (previously Resales), which include products that are bought from other establishments or transferred from other establishments of your company and then sold without further manufacture, processing, or assembly by this establishment. Report Wholesale products in any relevant prelisted product code, click the "Add Product Not Listed" button and search for an existing Wholesale product, or use the section for "Add product(s) not listed above."

Products made from materials owned by others (i.e., the customer). Report your commission or contract receipts in the appropriate

Contract Manufacturing product line(s).

Freight charged

Excise taxes

1) 2017900000 - Manufacturing of nonelectric breathing devices (including N95 and other respirators), incubators, inhalators, and resuscitators, and other surgical and medical apparatus and instruments, excluding anesthetic apparatus and parts

2) 2018000000 - Manufacturing of electromedical equipment (including diagnostic, therapeutic, patient monitoring equipment, and ventilators), excluding ionizing radiation equipment

3) 2050375000 - Manufacturing of personal safety equipment and clothing, industrial and nonindustrial, including respiratory protection, face shields, masks, and protective clothing, excluding footwear, gloves, and surgical and medical respirators

4) 2045875000 - Manufacturing of surgical appliances and supplies, including surgical gloves, bandages, gauze, cotton (sterile and non-sterile), and other surgical dressings, excluding orthopedic and prosthetic appliances

5) 2010475000 - Manufacturing of bath, facial, and hand soaps, including hand sanitizers

6) 2007875000 - Manufacturing of other household specialty cleaning and sanitation products, including disinfectants

• Do any of your manufacturing establishments produce the products listed in lines 1-6?

• Did you produce these product(s) prior to 2019?

  • If no, why did you begin producing these product(s)?

• Are you tracking the detailed sales/receipts/revenue for these new products?

• What would you have to do in order to gather that data for these new products?

• Are there any terms that you aren’t familiar with?

• Are there any other new products that any of your establishments have begun producing in 2020 because of the pandemic that aren’t listed above?

• Are there any products that you are planning to begin producing because of the pandemic that is different than what you have historically manufactured?

Wrap-up

We are almost done, and I just have a few wrap-up questions.

• Do you think this question would be easy to answer, difficult to answer, or somewhere in between? Why?

• Is there anything else we haven’t touched on today, that you think is relevant?

• Do you have any questions for us?

Thank you very much for your time today!

Questions

1. How many days during 2020 was your establishment closed due to the coronavirus pandemic?

Q1 (January 1, 2020 – March 31, 2020)

Q2 (April 1, 2020 – June 30, 2020)

Q3 (July 1, 2020 – September 31, 2020)

Q4 (October 1, 2020 – December 31, 2020)

2. ITEM 5: SALES, SHIPMENTS, RECEIPTS, OR REVENUE

A. What was the total value of products shipped and other receipts for this establishment? (Report detail in item 22)

Include:

o All products physically shipped from this establishment during 2020

o Products donated and physically shipped from this establishment during 2020

Exclude:

o Freight charges

o Excise taxes

3. Did this establishment donate any products during 2020?

Yes

No

B. What was the value of the donated products? (This is a breakout of the $,000 reported in Item 5, line A.)

ITEM 22: DETAIL OF SALES, SHIPMENTS, RECEIPTS, OR REVENUE

Of the $,000.00 of Sales, Shipments, Receipts, or Revenue reported in Item 5, what was the value of each product or service?

General – Please do not combine product lines. If the information is not directly available from your records, reasonable estimates are

acceptable.

The manufactured products and services listed below are generally made in your industry. If you make products or have revenue from sources not

listed, click the “Add Product Not Listed” button and search for an existing product, or use the section for “Add product(s) not listed above.”

Manufacturing of Products – Report the value of the products shipped and services performed at the net selling value, free on board (FOB) plant to

the customer, after discounts and allowances.

Include:

Products made elsewhere by others from materials supplied by this establishment. Report these products on the specific lines as if they were

made in this establishment.

Products transferred to other establishments within your company. These products should be assigned the full economic value (market value);

i.e., include all direct costs of production and a reasonable proportion of all other costs and profits.

Exclude:

Wholesale products (previously Resales), which include products that are bought from other establishments or transferred from other

establishments of your company and then sold without further manufacture, processing, or assembly by this establishment. Report

Wholesale products in any relevant prelisted product code, click the "Add Product Not Listed" button and search for an existing Wholesale

product, or use the section for "Add product(s) not listed above."

Products made from materials owned by others (i.e., the customer). Report your commission or contract receipts in the appropriate

Contract Manufacturing product line(s).

Freight charged

Excise taxes

1) 2017900000 - Manufacturing of nonelectric breathing devices (including N95 and other respirators), incubators, inhalators, and resuscitators, and other surgical and medical apparatus and instruments, excluding anesthetic apparatus and parts

2) 2018000000 - Manufacturing of electromedical equipment (including diagnostic, therapeutic, patient monitoring equipment, and ventilators), excluding ionizing radiation equipment

3) 2050375000 - Manufacturing of personal safety equipment and clothing, industrial and nonindustrial, including respiratory protection, face shields, masks, and protective clothing, excluding footwear, gloves, and surgical and medical respirators

4) 2045875000 - Manufacturing of surgical appliances and supplies, including surgical gloves, bandages, gauze, cotton (sterile and non-sterile), and other surgical dressings, excluding orthopedic and prosthetic appliances

5) 2010475000 - Manufacturing of bath, facial, and hand soaps, including hand sanitizers

6) 2007875000 - Manufacturing of other household specialty cleaning and sanitation products, including disinfectants