Draft 5/7/15
2016 Report of Organization –
Enterprise Statistics
Cognitive Interviews
DRAFT PROTOCOL
May 7, 2015
This protocol is a guide – the questions presented here will not necessarily be asked exactly as worded in the protocol or in this order. It is important to note that not all questions will be asked in every interview.
NOTE: As much information about interview participants and enterprise/establishments should be obtained prior to the interview as possible. Conduct background research on each enterprise prior to each interview.
Research Questions:
Do respondents understand the questions and terminology?
Can respondents follow the skip instructions?
Is the requested information available in respondent’s records?
Do respondents understand the FGP concept, and do they have any FGP-related data available?
How do respondents react to the idea of “big data / data transfer”?
Expected length of interview: 60 minutes
Materials Needed:
Draft questions
Copies of draft Report of Organization, NC-99001
Consent forms
Digital recorder
General probes that may be used throughout the interview:
Reflect back on respondent’s answer: “You said…”
In your own words, what is this question asking?
How did you arrive at this number/answer this question?
What records (if any) did/would you look at? What line or lines were of interest?
Specifically, what did you include in your answer? What did you exclude?
Would you consult other people to obtain an answer to this question?
1. Introduce observers and their background
2. Purpose of visit:
Thank you very much for your time. Let me start by telling you a little about what we will be doing today. We have been reaching out to some respondents who completed the Report of Organization in the past. This survey is completed annually and collects information from businesses throughout the United States. We are in the process of redesigning portions of this form in order to better meet our data users’ needs. We would like to get your input on the revised questions, in order to make them as clear and easy to complete as possible.
I apologize in advance, but as we proceed, I may have to interrupt you and move on to another question, in order to make the best use of our limited time together. Please also be aware that some questions may sound repetitive or similar. I also want to encourage you to think aloud during our conversation and share your immediate thoughts and/or reactions. It is ok if you are not sure or do not know about some of the things we discuss. If this is the case just let me know.
3. Permission to audio record discussion? Have R sign consent form.
Before we get started, I would like to audio record this interview so I don't have to rely on my memory later. This session is voluntary and confidential. Only Census Bureau employees connected with this project will have access to your recording. If that is all right with you, please sign this consent form. It also tells you about the confidentiality of this session.
Before we start with more specific questions, I would like to ask you a few questions about this company and your role in it. For today, we will be focusing on this company --(NAME).
(For each person in the room) What is your formal job title?
Could you tell me a little about your role and responsibilities?
2. Enterprise Background
What would you consider to be the main activity of this company? (e.g., manufacturer, wholesaler, retailer, etc.?)
Does this company manufacture products?
Now I’d like to discuss several specific data items. It may be helpful to have your records available if they are accessible at this time.
Note: If a respondent answers “I don’t know” to any of these questions, follow up with these probes:
Who in your company would know about that?
What is their title?
Do they work at this location?
(If no) Where are they located?
1. Primary Enterprise Activity (1)
How did you come up with your answer to this question?
Are the categories clear to you? Are they mutually exclusive? Are any missing?
Would you consult any records to answer this question?
Would you consult anyone else in your company in order to answer this question?
Do you have any suggestions to improve this question?
(Watch to see how respondent follows the skip instructions.)
2. Total Revenue (2) (same for each of the options selected on pages 2-7)
In your own words, what is this question asking?
What does “total revenue / operating revenue / net sales” mean to you?
How did you arrive at this number/answer this question?
What records (if any) did/would you look at? What line or lines were of interest?
Specifically, what did you include in your answer? What did you exclude?
Would you consult other people to obtain an answer to this question?
Did you read the bulleted instructions below the question?
Are there any other instructions that would be helpful to you in answering this question?
How do you go about consolidating for all subsidiary establishments and enterprises located in the United States?
Was the include instruction in the first bullet referring to your company divisions helpful or not helpful?
What does the third bullet “Include sales to foreign parents, foreign subsidiaries, and foreign affiliates” mean to you?
Would you be including or excluding sales to US locations or markets? Would you be including or excluding sales at US locations?
3. Own, Lease or Control Intellectual Property (3)
In your own words, what is this question asking?
What does “intellectual property” mean to you?
What does the term “own, lease or control any intellectual property” mean to you?
How did you arrive at your answer to this question?
Would you consult other people to obtain an answer to this question?
Did you read the definitional instruction above the question?
Where would be the best location for a note like this?
Are there any other instructions that would be helpful to you in answering this question?
Is there anything unclear about this question?
(Watch to see how respondent follows the skip instruction.)
4. Revenue from Intellectual Property (4)
In your own words, what is this question asking?
What does “royalties and license fees” mean to you?
How did you arrive at this number/answer this question?
What records (if any) did/would you look at? What line or lines were of interest?
Specifically, what did you include in your answer? What did you exclude?
Would you consult other people to obtain an answer to this question?
Did you read the bulleted instructions below the question?
Could you explain what the last bullet is saying in your own words?
Are there any other instructions that would be helpful to you in answering this question?
Could you provide each line item, or only a total?
Please review each line and tell me if there are any lines that are unclear to you or could be clarified.
5. Operating Expenses for Establishments in the US (5)
(For 5a: ) In your own words, what is this question asking?
What does “operating expenses for establishments located in the United States” mean to you?
How did you arrive at your answer to this question?
Would you consult other people to obtain an answer to this question?
Are there any other instructions that would be helpful to you in answering this question?
Is there anything unclear about this question?
(For 5b: ) In your own words, what is this question asking?
How did you determine which category to select? Would it be easier or harder for you to report actual percentages?
What does “foreign sources, including foreign parents, foreign subsidiaries, and foreign affiliates” mean to you?
6. Cost of Goods Sold (Establishments in US) (6)
In your own words, what is this question asking?
What does “cost of goods sold” mean to you?
What does “sold by this company’s establishments located in the United States” mean to you?
What about “”including exports from U.S. establishments?”
How did you arrive at this number/answer this question?
What records (if any) did/would you look at? What line or lines were of interest?
Specifically, what did you include in your answer? What did you exclude?
Would you consult other people to obtain an answer to this question?
Did you read the bulleted instructions below the question?
Are there any other instructions that would be helpful to you in answering this question?
Is there anything unclear about this question?
7. Sell Goods at US locations Incorporating IP (controlled by Enterprise) (7)
In your own words, what is this question asking?
What does the term “incorporated intellectual property” mean to you?
How did you arrive at your answer to this question?
What products are you considering?
What records (if any) did/would you look at? What line or lines were of interest?
Specifically, what did you include in your answer? What did you exclude?
Would you consult other people to obtain an answer to this question?
Are there any other instructions that would be helpful to you in answering this question?
Is there anything unclear about this question?
(Watch to see how respondent follows the skip instruction.)
8. Cost of Goods Sold (incorporating IP) (8)
(For 8a: ) In your own words, what is this question asking?
How did you arrive at this number/answer this question?
What records (if any) did/would you look at? What line or lines were of interest?
Specifically, what did you include in your answer? What did you exclude?
Would you consult other people to obtain an answer to this question?
Did you read the bulleted instructions below the question?
Are there any other instructions that would be helpful to you in answering this question?
Is there anything unclear about this question?
How does this question relate to Question 6?
Could you tell me a little bit more about that?
(For 8b: ) In your own words, what is this question asking?
How did you arrive at your answers?
Did you see the “Estimates are acceptable” instruction below the lines?
(For 8b alternative: ) How did you determine which category to select? Would it be easier or harder for you to report actual percentages for each line?
Did you see the “Estimates are acceptable” instruction below the lines?
(For 8c: ) In your own words, what is this question asking?
Are there any other materials that you are considering?
Would you include or exclude those?
9. Manufacture Goods Incorporating Intellectual Property (9)
In your own words, what is this question asking?
What does “completely or partially manufacture goods” mean to you?
How did you arrive at your answer to this question? Would you consult other people to obtain an answer to this question?
Are there any instructions that would be helpful to you in answering this question?
Is there anything unclear about this question?
(Watch to see how respondent follows the skip instruction.)
10. Operating Revenue from Goods Incorporating Intellectual Property (10)
In your own words, what is this question asking?
How did you arrive at your answer to this question?
Would you consult other people to obtain an answer to this question?
Are there any instructions that would be helpful to you in answering this question?
How did you determine which category to select? Would it be easier or harder for you to report an actual percentage?
Did you see the “Estimates are acceptable” instruction below?
Is there anything unclear about this question?
Thank you for your help, we are almost done. We are trying to understand the relationship that manufacturing companies may have with contractors. Specifically, there are some types of companies who are responsible for the manufacturing of products – they control the design and all the materials used to make them – but they do not physically make the products themselves. This is a phenomenon that the US does not currently measure, but would like to, as it could be very informative about the US economy.
Thinking about what I just described, how would you describe your company? (a manufacturer, a manufacturer with others producing your products, a wholesaler, a service, etc.)
Where are the products you sell manufactured?
Are they manufactured by your company? Affiliates of your company? Other companies?
Are they manufactured in the US? Outside the US?
Is this for all or some of the products/product lines? Could you tell me a bit more about that?
Thinking about the products that are manufactured by other companies, where are these products designed?
Who owns/controls the IP for these products?
Is the IP owned/controlled by a specific establishment or plant? Headquarters? Other?
Which entity makes decisions about the materials that are being used to manufacture these products?
Your company? The manufacturer that is doing the work? Does your company control decisions on which materials are used even if you do not purchase the materials directly?
Which entity makes the decisions about where the product will be sold?
How about how much it will be sold for?
Are they sold in the US or to other countries?
If a product was defective or damaged, who would be responsible?
What if a product is not sold? Who takes the liability?
Could you tell me what the term “contract manufacturing” means to you?
Is this a term you have heard of?
Is it a term you use?
Do you have other terms to describe this concept?
Could you tell me what the term “material inputs” means to you?
Is this a term you have heard of?
Is it a term you use?
Do you have other terms to describe this concept?
How does “material inputs” differ from “materials”?
If we were to ask questions about this concept on some of our surveys where would be the best place to gather this information (e.g., the enterprise, specific establishments)
Do you have any records on this type of information?
Where would that information be located?
Would any specific locations/establishments have information on this topic?
Could you provide us with some (2 or 3) establishment contacts to follow up with and get their input on this topic?
Is there a central office or person that tracks information about the revenue produced by each major product line?
a. (If yes) Would you be able to provide us with a contact person at that location who might be able to help us answer our questions?
These questions will be added to the end of any qualitative research protocol that is used during visits with large or medium-sized companies (designated with an L, M, or N in the Business Register).
Finally, I want to briefly change topics and discuss with you the idea of transferring files between businesses. Have you heard of the term file transfers before today? (Yes/No)
Is this a term that is used within your organization? (Yes/No)
If no, what term is used?
Have you ever provided financial or other data to another party outside of your organization in the form of a file transfer
?
IF YES: Can you briefly tell us about your experiences with file transfers?
Probe if needed: what type of organization did you transfer the data to?
Probe if needed: what type of data did you send?
IF NO:
Are you familiar with the concept of a file transfer?
Do you know of anyone in your company who is involved with file transfers?
On scale of 1 to 5, with 1 being very unwilling and 5 being very willing, how willing would your company be to provide data to the Census Bureau in this manner, in lieu of completing a survey?
What software does your company use for your accounting and payroll needs?
Would you be willing to talk about file transfers with us in more detail at a later time?
Overall, what do you think of the questions we discussed today?
Do you have any other comments, questions or suggestions for us?
Thank you for your help today!
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Heidi M St.Onge |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |