2009 BRDIS Pre-Testing: Cognitive Interview Protocol

Status: Original and derived artifacts are available for this document.

Download: doc | pdf | html

2009 BRDIS Pre-Testing:  Cognitive Interview Protocol

General Probes:  
-	What is this item/section asking you for, in your own words?
-	What terms do you use to describe this?
-	Do your records contain this information? What do these records look like? What are they called?
-	Would you use book figures, or would you have to estimate to report for this item?
-	Would you have to manipulate the book figures to get an answer? How?
-	How would you go about completing this item? What specifically would you include (exclude)?
-	Would you need help from other company personnel to get these data?
-	You said, “____”?
-	Can you tell me more about that?
-	What are/were you thinking about?
-	(Reflect back R’s answer and ask R to correct)  I want to make sure I have it right. I think you said ____?

Background questions 
-	Please tell me a little about your company’s activities, and your role in the company.
-	Please tell me a little about your company’s R&D activities. 
-	Do you interact with your company’s R&D managers or other R&D personnel? Please tell me about your relationships with R&D personnel.


Now I will show you parts of next year’s survey form.  Please approach it as if you were about to fill it out.  I will ask you questions as you go through it.



Pages 2-3, General Information and Instructions  

Observe whether R attends to any information on this page, skims, looks briefly and goes on to page 4, etc.  

Specific probes:  
-	What do you notice about these pages?
-	What is the most important information to you on these pages?

-	(R at foreign-owned company)  What does this instruction mean to you?
	-	Are any of these instructions relevant for your company?


Page 4, Questions 1-3  

For each item:  
-	What does this item mean to you?
-	How would you go about completing this item?

Flowchart:  
-	Observe whether R attends to the flowchart, looks at it briefly and goes on, 
etc.  
-	What does this diagram mean to you? 
-	After looking at this diagram, what would you do next?


Page 5, questions 4-6  

For each item:  
-	What does this item mean to you?
-	How would you go about completing this item?


Specific probes:  

Q6:  What does “domestic operations” mean to you with regard to net sales or operating revenues? 
-	Would you report sales made IN the US, sales made BY US entities, or something else?


Page 6, questions 7-10  

For each item:  
-	What does this item mean to you?
-	How would you go about completing this item?


Page 7, questions 1.1-1.4  

For each item:  
-	What does this item mean to you?
-	How would you go about completing this item?


Specific probes:  

Q1.1:  
-	What does the text in the box mean to you?

Q1.2:  
-	Each line:  
-	What does this mean to you?  
-	Does it apply to your company?
-	What terminology do/would you use to describe this?

Q1.4:  
-	Note whether R observes skip instruction.  
-	If appropriate, skip to section 2 on page 16.  


Page 8, R&D paid for by your company  

For each item:  
What does this item mean to you?
-	How would you go about completing this item?


Specific probes:  

Q1.5:  
-	What does the text in the box mean to you?
	-	Would you have to estimate these amounts? If so, how would you go 
about it?

Observations:
-	Note whether R sees the reference to codes on page _.  
-	Note whether R observes instruction “total equals question 1.3”

Q1.6:  
-	What would you report in line q., “all other”?


Page 9, questions 1.7-1.10  

For each item:  
-	What does this item mean to you?
-	How would you go about completing this item?


Specific probes:  

Q1.7:  
-	What does “domestic R&D paid for by your company” mean to you?

Q1.8:  
-	What does “total foreign R&D paid for by your company” mean to you?

Q1.9:  
-	How would you go about completing columns 1, 2, and 3?
-	What does “payments to collaborators” in line i. mean to you? 
-	Does it apply to your company? 
-	What terminology do/would you use to describe this?
-	What does “purchased R&D services” in line j. mean to you?
-	Does it apply to your company? 
-	What terminology do/would you use to describe this?

Q1.10:  
-	How would you go about completing columns 1, 2, and 3?
-	What does “R&D performed by others” mean to you? 
-	Does it apply to your company? 
-	What terminology do/would you use to describe this?


Page 10-11, questions 1.11-1.12  

For each item:  
What does this item mean to you?
-	How would you go about completing this item?


Specific probes:  

Q1.11:  
-	What does “domestic R&D performed by your company” mean to you?


Page 11, questions 1.13-1.15  

For each item:  
-	What does this item mean to you?
-	How would you go about completing this item?


Page 12-13, questions 1.16-1.25  

For each item:  
-	What does this item mean to you?
-	How would you go about completing this item?


Specific probes:  

Q1.16:  
-	What does “domestic R&D expense” mean to you?
-	Is domestic R&D expense something that exists in your records?
	-	If no:  How would you go about generating this number?
-	What does “paid for by your company’s operations irrespective of where the R&D was performed” mean to you?

Q1.17:  
-	In your opinion, what is the difference between columns 1 and 2?


Page 14, Section 2, Financial Schedule 2  

For each item:  
What does this item mean to you?
-	How would you go about completing this item?


Specific probes:  

Q2.1:   
-	What does “costs for R&D funded, paid for, or reimbursed by others not owned by your company” mean to you?
-	Does it apply to your company? 
-	What terminology do/would you use to describe this?
-	Each line:  
-	What does this mean to you?  
-	Does it apply to your company?
-	What terminology do/would you use to describe this?

Q2.3:  
-	Note whether R observes skip instruction.  
-	If appropriate, skip to wrap-up questions.  


Page 15, questions 2.4-2.6  

For each item:  
-	What does this item mean to you?
-	How would you go about completing this item?


Specific probes:  

Q2.4:  
-	What do these amounts represent? 
-	(if clarification needed)  Do they represent work performed in these locations, locations of others who are paying you to do work, or something else?

Q2.5:  
-	What does “domestic R&D paid for by others” mean to you?

Q2.6:  
-	What does “foreign R&D paid for by others” mean to you?


Page 16-17, questions 2.7-2.11  

For each item:  
-	What does this item mean to you?
-	How would you go about completing this item?


Specific probes:  

Q2.7:  
-	What does “payments to collaborators” in line i. mean to you? 
-	Does it apply to your company? 
-	What terminology do/would you use to describe this?
-	What does “purchased R&D services” in line j. mean to you?
-	Does it apply to your company? 
-	What terminology do/would you use to describe this?

Q2.8:  
-	What does “domestic R&D performed by others that was paid for by others” mean to you?

Q2.9:
-	What does “domestic R&D performed by your company that was paid for by others” mean to you?
-	In your opinion, what is the difference between 2.8 and 2.9?

Q2.10:  
-	What does the text in the box mean to you?
	-	Would you have to estimate these amounts? If so, how would you go 
about it?

Observations:
-	Note whether R sees the reference to codes on page _.  
-	Note whether R observes instruction “total equals question 2.2”

Q2.11:
-	Note whether R observers the switch to domestic?


Pages 18-21, questions 2.12-2.18  

For each item:  
-	What does this item mean to you?
-	How would you go about completing this item?


Specific probes:  

Q2.12:  
-	What does the text in the box mean to you?
	-	What does “subcontractor” mean to you?
-	What does “subgrantee: mean to you?
	-	What does “original source of funds” mean to you?

Q2.13:  
-	How well does this amount match what you carry in your records?

Q2.18:
-	What does each column mean to you?
-	Note whether R observes that this is for domestic R&D


Page 22, questions 2.19-2.24  

For each item:  
What does this item mean to you?
-	How would you go about completing this item?


Wrap-up  

How long do you think it would take you to complete the questions we reviewed today?

How easy or difficult would it be to complete these questions – very difficult, somewhat difficult, neither easy nor difficult, somewhat easy, very easy?