BRDIS RD Protocol draft 3-24-09
Business R&D and Innovation Survey – Respondent Debriefing Protocol
Research questions:
- Does the information reported by companies match the intent of the BRDIS?
- What is the composition of reported data?
- What types of unintended information are reported, if any?
- What types of desired information are omitted, if any?
- Does the information requested by BRDIS exist within companies, and do coordinators and respondents have knowledge of and access to existing information?
- Are appropriate personnel completing the sections of the survey and/or providing assistance?
- Survey coordinators:
- How do survey coordinators interpret the sections of the form?
- How do they find and recruit respondents (or local data providers) for each section? Which sections do they take responsibility for completing themselves?
- What communications take place between coordinators, respondents and local data providers? What information is shared and how?
- How do coordinators coordinate responses from respondents and submit the information to Census?
- How do coordinators and respondents interpret the survey concepts, especially:
- R&D activities
- Business codes
- R&D expense vs. R&D performance
- R&D performed by others
- R&D funded/paid for by others
- Geographic concepts (worldwide vs. domestic, country, US state)
- R&D employees, and breakouts by occupation, gender, etc.
- R&D FTEs
- Innovation
- What is the level of burden imposed by BRDIS response in terms of time, effort, and number of personnel involved, etc.?
General Probes:
- What is this item/section asking you for, in your own words?
- Did you have to manipulate book figures to get an answer for this item? How?
- Did you have to estimate to report for this item? If so, what estimation method did you use? In your opinion, how close is the estimated figure to an actual figure, as a percentage?
- How did you go about completing this item?
- What specifically did you include in your answer?
- How did you decide what to include?
- Did you specifically exclude anything? If so, what?
- How did you decide what to exclude?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- Did you need help from other company personnel to get these data?
- You said, “____”?
- Can you tell me more about that?
- What are/were you thinking about?
- (Reflect back R’s answer and ask R to correct) I want to make sure I have it right. I think you said ____?
Procedures:
For each section, first probe the highest priority survey items. Then probe all other survey items with positive response. Probe response process issues as time permits.
Every coordinator and/or Section 2 respondent should be asked questions about the Section 4 instructions and question 4.1 if they reported 0 for 4.1 (funded R&D). See specific instructions for “0” cases on page 3 and page 16.
Part I: Survey coordinator
- Did you complete any portions of the survey yourself? Which ones?
- Coordinator is R – Go to questions for BRDIS sections completed by coordinator, then return to the coordinator questions below
- Please tell me about how you handled the Business R&D Survey, the process you went through in completing it.
- What do the parts of the form mean to you? What kinds of information do they ask for?
- Funded R&D “0” cases – Go to specific questions about section 4 instructions and Q4.1 on page 16 at this point.
- Did you get help from other people in the company?
- Who were they?
- How did you go about finding people to assist you?
- How did they help you (ie, answer specific questions, take over sections,
etc)?
- Which sections did they assist with?
- In your opinion, do you think you were able to get help from the most
appropriate people in the company?
- (yes or no) Why do you think they [were/were not] the most appropriate?
- (if no) Do you know of any company personnel or areas that would
have been more appropriate for those sections?
- Who are they?
- Why would they have been more appropriate, in your opinion?
- Did you seek assistance from these people or areas?
- (if yes) In your opinion, what prevented you from getting their
assistance?
- Did you pass along any information about the survey to the people who were assisting you?
- (if yes) What did you give them? How did you decide what information to
share with them?
- Did they have any questions for you, once they received your request?
What did they ask? How did you respond?
- Do you recall reading the survey’s definition of R&D on page 4?
- (if yes) Did this definition influence the way you completed the form?
- Did you find it necessary to share this definition with any of the people
who were assisting you with the form?
- How did you make the actual survey request to other people? E.g., did you make phone calls, send emails, photocopies of the form, etc?
- How did you get the requested data from those who assisted you?
- How did you report it to the Census Bureau?
- (if did not use Census Taker) Were you aware of the option to report via Census’ website?
- (if yes) What made you decide to return your survey data through the
mail/fax rather than through the website?
- (if used CT) What made you decide to return your survey data through
the website?
- Overall, were you satisfied or not satisfied with your experience of
using the website? How would you suggest we make it better?
- Do you recall seeing portions of the survey form in spreadsheet form on
the Census Bureau website? Did you use them?
- (yes or no) What made you decide to [use/not use] them?
- (if used them) Did you find them useful or not useful? How would you
suggest we make them better?
- Were you the contact person for the previous R&D form, the Survey of Industrial R&D?
- (if no) How were you selected to handle the new Business R&D and
Innovation Survey?
- Did you use the Census Bureau website to update your contact information?
- (if yes) What made you decide to use the website?
- Did you find that option useful or not useful?
- Were there any other features or information you checked out on the website? What were they? Did you find them helpful or not helpful?
Part II: Section 1, Company Information
Highest priority item:
1.5
- What does “business code” mean to you in the context of this item?
- How did you decide to keep/change these codes?
- (Note whether R selected new codes)
- Did you review the list of codes on pages 8-9?
- (if yes) How did you go about reviewing the list of codes?
- (if R selected new code/s) How did you decide which code(s) to choose?
- How well do our codes match the way you think about your company’s
activities?
- Did you provide the codes to other people?
- (if yes) How did you go about that?
- What do you think of the text in the lighter colored box? Do you find that useful or not useful?
- What does “your company” mean to you in the context of this item?
Lower-priority items – Probe remaining questions as time permits.
For each item (as applicable):
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
Specific probes:
1.1
- How well does this describe your role in preparing and submitting this survey for your company? Is there anything we left out?
1.8
- Row i., “less eliminations” – What does this line mean to you? How did you handle it?
Part III: Section 2, R&D Paid for by Your Company (R&D Expense)
About the respondent (if different from coordinator):
- What is your role in the company? What are your responsibilities?
- Where in the company are you located? (I.e., at corporate level, division level, subsidiary, operating unit, R&D center, etc.)
- What is your relationship to your company’s R&D activities?
- Do you interact with managers or executives who oversee R&D activities, or their staff?
- (if yes) Please describe these interactions.
Let’s take a look at section 2. Please tell me in your own words what section 2 is about. What kinds of information does it ask for?
Now let’s look at some specific items. We’ll skip around to make sure we cover the high-priority items, then go back over some others if there is time.
Highest priority items
Instructions
- Do you recall reading the text on page 12?
- Is there anything on that page that influenced the way you completed the form?
- If so, what? How did it influence the way you completed the form?
- What does “R&D performed by your company” mean to you?
2.1
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- How did you decide what to include?
- Did you specifically exclude anything? If so, what?
- How did you decide what to exclude?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- Do you recall reading the text in the lighter colored box?
- What does each bullet mean to you?
- Costs for routine product testing – Did your company have those?
- If so, are they associated with R&D projects?
- Are they included in your answer?
- Do you find these instructions helpful or not helpful?
- Did the instructions influence the way you answered this question? If so, how?
- What does “your company” mean to you in the context of this item?
- Did you file a 10K for 2008? If so, how well does your answer to this question relate to what you reported for R&D expense in your 10K report?
2.3
- What does this question mean to you?
- How did you go about answering it?
- Line a: What does “performed by your company” mean, in your opinion?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- What does the text in the lighter colored box mean to you?
- Did this instruction influence the way you completed this item? If so, how?
- Line b: What does “performed by others not owned by your company” mean to you?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Does your answer include all contract services that are part of your R&D
expense, or only some of them? If only some of them, how did you decide
what to include and what to exclude?
- What does the text in the lighter colored box mean to you?
- Did this instruction influence the way you completed this item? If so, how?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
2.6
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- How did you decide what to include?
- Did you specifically exclude anything? If so, what?
- How did you decide what to exclude?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- Do you recall reading the text in the lighter colored box?
- What does it mean to you?
- Do you find this information helpful or not helpful?
- Did this instruction influence the way you completed this item? If so, how?
- What does “domestic” mean to you in the context of your company?
- In your opinion, is this question related to question 2.3? If so, how?
- Do the amounts you reported include R&D that your company paid another company to perform?
- Do the amounts you reported reflect the actual locations where R&D was performed, or the locations of companies carrying the expenses on their books and not necessarily locations where R&D was performed?
- (if non-0) Line q., “all other” – What countries are included here?
2.7 (Only probe for non-0’s, and for cases known to have foreign subsidiaries but reporting 0)
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- How did you decide what to include?
- Did you specifically exclude anything? If so, what?
- How did you decide what to exclude?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
2.8
- Does your company have consultants or other personal service contract costs as part of R&D expense? If yes, where would those costs be reported?
- (If non-0 for line i., All other costs) What is included in this figure?
- (for multi-national companies) Compared to global R&D expense, is it any more or less difficult to provide this breakout for your company’s domestically performed R&D?
2.12
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- How did you decide what to include?
- Did you specifically exclude anything? If so, what?
- How did you decide what to exclude?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- Do you recall reading the text in the lighter colored box?
- What does it mean to you?
- Do you find this information helpful or not helpful?
- Did this instruction influence the way you completed this item? If so, how?
- How well do the business codes match your records related to R&D expense?
- Were there any R&D expense amounts that were not included in any of these codes?
- (If R is not coordinator) How did you get the codes reported here?
2.13
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- How did you decide what to include?
- Did you specifically exclude anything? If so, what?
- How did you decide what to exclude?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- How well does this question match your records?
- In your opinion, is this question related to question 2.6, line a? If so, how?
2.14
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- In your opinion, what is the difference between columns 1 and 2?
- How well does this item match the way your records are kept?
2.15
- What does this question mean to you?
- What does “capital expenditures for R&D operations” mean to you in the context of your company?
- How did you go about answering this question?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did your company purchase land in 2008? If so, is that included in this item?
- Did your company purchase any patents or other intangible assets in 2008? If so, is that included in this item?
2.16
- What does this question mean to you?
- How did you go about answering this question?
- (If company purchased land and/or intangible assets in 2008)
- On which line did you report the land your company purchased in 2008?
- On which line did you report the intangible assets your company
purchased in 2008?
- (If company purchased no land or intangible assets in 2008)
- Where do you think you would report purchase of land and intangible
assets if you had them?
- How well do these categories match your records related to capital expenditures?
Lower-priority items – Probe remaining non-0 responses as time permits.
For each item:
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
Specific probes:
2.5
- What does each line mean, in your opinion?
- How well do these categories match your records?
2.9
- What does this question mean to you?
- How did you go about answering it?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- Do the amounts you reported reflect the actual locations where R&D was performed, or the locations of companies carrying the expenses on their books and not necessarily locations where R&D was performed?
2.17
- What does the text in the box under the question mean to you?
- Do you find that information useful or not useful?
- Did this instruction influence the way you completed this item? If so, how?
- In your opinion, what is each line asking for?
- How well does this item match the way your records are kept?
2.19
- How did you go about answering this question?
- Did your response originally include a fraction of an hour? If so, how did
you decide what to report?
- What kinds of activities did you consider?
- Does this answer include only your time, or also time spent by others?
2.20
- Are there any people who assisted with preparation of this section of the survey who are not listed here? How many? What are their titles?
Wrap-up
- In your opinion, are you the most appropriate person in the company to complete this section of the survey?
- (yes or no) What makes you [not] the best person to complete this
section?
- Did you need help from others for any parts of this section?
- (if yes) In your opinion, were you able to get help from the most appropriate people in the company?
- (yes or no) Why do you think they [were/were not] the most appropriate?
- (if no) Do you think there are any company personnel or areas that would
have been more appropriate for those sections?
- Who are they?
- Why would they have been more appropriate, in your opinion?
- Did you seek assistance from these people or areas?
- (if yes) In your opinion, what prevented you from getting their assistance?
Part IV: Section 3, R&D Management and Strategy
About the respondent (if different from coordinator):
- What is your role in the company? What are your responsibilities?
- Where in the company are you located?
- What is your relationship to your company’s R&D activities?
- Do you interact with managers or executives who oversee R&D activities, or their staff?
- (if yes) Please describe these interactions.
Let’s take a look at section 3. Please tell me in your own words what section 3 is about. What kinds of information does it ask for?
Now let’s look at some specific items. We’ll skip around to make sure we cover the high-priority items, then go back over some others if there is time.
Highest priority items:
Cover
- Do you recall reading the text on page 21?
- If so, is there anything on that page that stood out to you?
- What does “strategic aspects of your company’s R&D” mean to you?
- What did you think of our use of this term?
Instructions
- Do you recall reading the text on page 22?
- Is there anything on that page that helped you in completing the form?
- If so, what? How did it influence the way you completed the form?
- What does “R&D performed by your company” mean to you?
- What does “your company” mean to you in the context of this section?
3.1-3.3
- (If R is different from the coordinator/section 2 R) How did you get the figures reported here?
- (if R provided the figures, not coordinator/section 2 R)
- (for each question) In your opinion, is the figure you reported the same as what would be reported by someone in corporate finance?
- (if R received the figures from coordinator/section 2 R)
- Did the person who gave you the form include these figures, or did you
have to ask him/her for them?
- Did you have any kind of discussion with that person about what these
figures represent?
- If so, what kinds of questions did you have?
- How did your colleague respond?
3.4-3.7
- (each question) What does this question mean to you?
- (if left blank) It appears you left this item blank; does that mean that the answer would be 0, or were you unable to answer this question for some reason?
- (if unable to answer) What is the reason you were unable to answer this
question?
- (if not left blank) How did you go about answering it?
- What specifically did you include in your answer?
- How did you decide what to include?
- Did you specifically exclude anything? If so, what?
- How did you decide what to exclude?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- 3.4: What does “business area or product lines” mean to you?
- What does “in which your company had no revenues before 2008” mean
to you?
- What kinds of activities are included in your answer?
- 3.5: “What does “science/technology that your company had not used in
projects before 2008” mean to you?
- What kinds of activities are included in your answer?
- 3.7: What does “R&D expense in science/technology new to the markets in which your company operates” mean to you?
- What kinds of activities are included in your answer?
- What does “first-to-market” mean to you?
- What does the exclude instruction mean to you, in your own words?
- These two terms, “new to the markets in which your company operates” and “first-to-market,” are they the same or are they different? If different, how?
- Did you find the examples in these questions useful or not useful?
- Did any of them influence the way you completed any item?
- How did you arrive at these percentages, that is, what did you divide by what?
- How well do your records support these particular questions?
- How easy or difficult was it to provide these answers? How long did it take you to answer these questions?
- How confident are you that the figures you reported match what is being asked for?
3.8
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- How did you decide what to include?
- Did you specifically exclude anything? If so, what?
- How did you decide what to exclude?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- What does “R&D agreement” mean to you in the context of this question?
- (if Federal contractor) Would you consider government contracts to be
R&D agreements?
- Did the definition influence the way you answered this question?
3.16-3.17
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- How did you decide what to include?
- Did you specifically exclude anything? If so, what?
- How did you decide what to exclude?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- 3.16: What does “research” mean to you?
- “Development”?
- Did the definitions of research and development influence the way you answered this question?
- How well do these categories match your records?
- What does “domestic” mean to you? Did that term influence the way you
answered this question?
- How easy or difficult was it to answer this question?
- Did your company perform R&D in 2008 that was funded or paid for by
others? If so, is that included in your answer?
- 3.17: What does “applied research” mean to you?
- “Basic research”?
- Did the definitions of basic and applied research influence the way you answered this question?
- How well do these categories match your records?
- How easy or difficult was it to answer this question?
- If we asked for this breakout by percentages of R&D personnel involved
rather than dollar amounts, would that be any easier or more difficult?
Lower-priority items – Probe remaining non-0 responses as time permits.
For each item:
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
Specific probes:
3.13
- (non-0 cases) Did your company renew any previously existing R&D agreements in 2008? If so, are they included in the number reported here?
3.22-3.35
- (for each) How well does this question match your understanding of your company’s R&D activities?
- How well does this question match your company’s records?
- 3.24: Did you find the example useful or not useful? Did it influence the way you answered this question?
- 3.35: What does “social science R&D” mean to you?
3.43
- How did you go about answering this question?
- Did your response originally include a fraction of an hour? If so, how did
you decide what to report?
- What kinds of activities did you consider?
- Does this answer include only your time, or also time spent by others?
3.44
- Are there any people who assisted with preparation of this section of the survey who are not listed here? How many? Who (titles)?
Wrap-up
- Overall, how confident are you that the information you provided matches what the survey is asking for? Which questions are you most confident about? Which questions are you least confident about?
- Overall, how easy or difficult was this section to complete? Which questions were most difficult in your opinion? Which ones were least difficult?
- In your opinion, are you the most appropriate person in the company to complete this section of the survey?
- (yes or no) What makes you [not] the best person to complete this
section?
- Did you need help from others for any parts of this section?
- (if yes) In your opinion, were you able to get help from the most appropriate people in the company?
- (yes or no) Why do you think they [were/were not] the most appropriate?
- (if no) Do you think there are any company personnel or areas that would
have been more appropriate for those sections?
- Who are they?
- Why would they have been more appropriate, in your opinion?
- Did you seek assistance from these people or areas?
- (if yes) In your opinion, what prevented you from getting their
assistance?
Part V: Section 4, R&D Funded or Paid for by Others
NOTE – Every coordinator and/or Section 2 respondent should be asked questions about the Section 4 instructions and Q4.1 regardless of whether they completed this section.
About the respondent (if different from the coordinator):
- What is your role in the company? What are your responsibilities?
- Where in the company are you located?
- What is your relationship to your company’s R&D activities?
- Do you interact with managers or executives who oversee R&D activities, or their staff?
- (if yes) Please describe these interactions.
Let’s take a look at section 4. Please tell me in your own words what section 2 is about. What kinds of information does it ask for?
Now let’s look at some specific items. We’ll skip around to make sure we cover the high-priority items, then go back over some others if there is time.
Highest priority items:
Instructions – (Ask for 0 cases and non-0 cases)
- Do you recall reading the text on page 30?
- Is there anything on that page that helped you in completing the form?
- What does “funded R&D” mean to you?
- (non-0 cases only) How well does this explanation match the way you think about your company’s R&D activities?
- What does “your company” mean to you in the context of this section?
4.1 - Note separate questions for 0 cases and non-0 cases
(0 cases)
- What does this question mean to you?
- How did you go about answering it?
- Do you recall reading the text in the lighter colored box?
- What do each of the bullets mean to you?
- Did these instructions influence the way you answered this question?
- Has your company ever received funds for R&D work from business partners?
- (if yes) How can we modify the instructions to make it clearer that these
amounts should be reported here?
- How well do these instructions match your company’s R&D activities?
(Non-0 cases)
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- Do you recall reading the text in the lighter colored box?
- What do each of the bullets mean to you?
- Did these instructions influence the way you answered this question?
- Did your company receive funds in 2008 for R&D work from business partners?
- (if yes) Are these amounts included here?
- (if no) How can we modify the instructions to make it clearer that these
amounts should be reported here?
- How well do these instructions match your company’s R&D activities?
- How well do they match your records pertaining to R&D?
- Do you find them useful or not useful?
4.2
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- How well does this question match your company’s records?
4.4
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- What does “domestic” mean to you in the context of your company?
- In your opinion, is this question related to question 4.2? If so, how?
- Do the amounts you reported include R&D that your company paid another company to perform?
- Do the amounts you reported reflect the actual locations where R&D was performed? (pause for answer, then clarify) Or the locations of companies carrying the costs on their books and not necessarily locations where R&D was performed?
4.5
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- (if R is not also the coordinator) Did the person who gave you the form include these codes, or did you have to ask him/her for them?
- Did you have any kind of discussion with that person about what these
codes represent?
- If so, what kinds of questions did you have?
- How did your colleague respond?
- How well do these codes match the way you think about your company’s activities?
- How well does this question match your records?
- In your opinion, is this question related to question 4.4? If so, how?
- Do you recall reading the text in the lighter colored box?
- What does it mean to you?
- Did it influence the way you answered this question?
- Do you find this information useful or not useful?
4.7
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- What does ___ mean to you?
- “Contracts”
- “Direct or prime contracts”
- “Subcontracts”
- “Grants”
- “Reimbursements”
- “All other agreements”
- How well do these categories match your records?
4.8
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- Does your company perform R&D as a subcontractor on Federal contracts awarded to other companies?
- If so, how did you report those R&D costs?
- How do those costs appear in your records?
- (Federal contractors) Has your company ever received a Small Business Innovation Research (SBIR) grant?
- If so, ...?
4.12
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
4.16-4.17
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- 4.16: What does “research” mean to you?
- “Development”?
- Did the definitions of research and development influence the way you answered this question?
- How well do these categories match your records?
- How easy or difficult was it to answer this question?
- 4.17: What does “applied research” mean to you?
- “Basic research”?
- Did the definitions of applied and basic research influence the way you answered this question?
- How well do these categories match your records?
- How easy or difficult was it to answer this question?
4.25-4.27
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- How well does this question match your company’s records?
Lower-priority items – Probe remaining non-0 responses as time permits.
For each item:
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
Specific probes:
4.8-4.10
- How well do these categories match your records?
4.11
- What does “independent research and development” mean to you?
4.13-4.15
- What does “projected R&D costs paid for by others” mean to you?
- How did you go about reporting [each line]?
4.18-4.24
- (for each) How well does this question match your understanding of your company’s R&D activities?
- How well does this question match your company’s records?
- 4.22: Did you find the example useful or not useful? Did it influence the way you answered this question?
4.29
- How did you go about answering this question?
- Did your response originally include a fraction of an hour? If so, how did
you decide what to report?
- What kinds of activities did you consider?
- Does this answer include only your time, or also time spent by others?
4.30
- Are there any people who assisted with preparation of this section of the survey who are not listed here? How many? Who (titles)?
Wrap-up
- Overall, how confident are you that the information you provided matches what the survey is asking for? Which questions are you most confident about? Which questions are you least confident about?
- Overall, how easy or difficult was this section to complete? Which questions were most difficult in your opinion? Which ones were least difficult?
- In your opinion, are you the most appropriate person in the company to complete this section of the survey?
- (yes or no) What makes you [not] the best person to complete this
section?
- Did you need help from others for any parts of this section?
- (if yes) In your opinion, were you able to get help from the most appropriate people in the company?
- (yes or no) Why do you think they [were/were not] the most appropriate?
- (if no) Do you think there are any company personnel or areas that would
have been more appropriate for those sections?
- Who are they?
- Why would they have been more appropriate, in your opinion?
- Did you seek assistance from these people or areas?
- (if yes) In your opinion, what prevented you from getting their
assistance?
Part VI: Section 5, R&D Human Resources
About the respondent (if different from the coordinator):
- What is your role in the company? What are your responsibilities?
- Where in the company are you located?
- What is your relationship to your company’s R&D activities?
- Do you interact with managers or executives who oversee R&D activities, or their staff?
- (if yes) Please describe these interactions.
Let’s take a look at section 5. Please tell me in your own words what section 5 is about. What kinds of information does it ask for?
Now let’s look at some specific items. We’ll skip around to make sure we cover the high-priority items, then go back over some others if there is time.
Highest priority items:
Instructions
- Do you recall reading the text on page 44?
- Is there anything on that page that helped you in completing the form?
- Were you aware of the definition of R&D on page 4 of the form?
- If so, did that instruction influence the way you completed this section?
5.1
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- What does this figure represent? That is, is it a head count at a particular period of time, or an average, or something else?
- (if head count) What date does this figure come from?
- Do you recall reading the text in the lighter colored box?
- What do each of these bullets mean to you?
- Did this instruction influence the way you answered this question?
- What does “your company” mean to you in the context of this item?
5.2
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- Was it necessary to get assistance from company personnel in other locations to report this figure?
- (if yes) Who were they? Where were they located?
- What does “R&D employees” mean to you?
- What do each of these bullets mean to you?
- Did this instruction influence the way you answered this question?
- How well do these instructions match the way you think about the employees associated with your company’s R&D activities?
- How well do they match the way you keep records?
5.6
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- How well does this question match your company’s records?
5.7
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- Was it necessary to get assistance from company personnel in other locations to report these figures?
- (if yes) Who were they? Where were they located?
5.8
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- How well does this question match your company’s records?
- Was it necessary to get assistance from company personnel in other
locations to report these figures?
- (if yes) Who were they? Where were they located?
5.10
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- Do you recall reading the text in the lighter colored box? What does it mean to you?
- How well do these instructions match your records pertaining to R&D personnel or activities?
- Do these instructions influence the way you answered this question?
- Does your company track any other kinds of information about the work of R&D personnel that might be easier for you to report?
5.11
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- How well does this question match your records?
Lower-priority items – Probe remaining non-0 responses as time permits.
For each item:
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
Specific probes:
5.3
- For lines a through d, what does each line mean to you?
- How well do these categories match the way you think about the
employees associated with your company’s R&D activities?
- Was it necessary to get assistance from company personnel in other locations to report these figures?
- How well do these categories match your company’s records?
- What are you reporting in columns 1 and 2?
- How well do these categories match your company’s records?
5.4
- For lines a and d, what does each line mean to you?
- In your opinion, is this question related to question 5.3? If so, how?
- How well do these categories match the way you think about your company’s
scientists and engineers?
- How well do these categories match your company’s records?
5.5
- What does “domestic” mean to you in the context of this question?
- How well does this question match your company’s records?
5.9
- What does “temporary visa” mean to you?
5.13
- How did you go about answering this question?
- Did your response originally include a fraction of an hour? If so, how did
you decide what to report?
- What kinds of activities did you consider?
- Does this answer include only your time, or also time spent by others?
5.14
- Are there any people who assisted with preparation of this section of the survey who are not listed here? How many? Who (titles)?
Wrap-up
- Overall, how confident are you that the information you provided matches what the survey is asking for? Which questions are you most confident about? Which questions are you least confident about?
- Overall, how easy or difficult was this section to complete? Which questions were most difficult in your opinion? Which ones were least difficult?
- In your opinion, are you the most appropriate person in the company to complete this section of the survey?
- (yes or no) What makes you [not] the best person to complete this
section?
- Did you need help from others for any parts of this section?
- (if yes) In your opinion, were you able to get help from the most appropriate people in the company?
- (yes or no) Why do you think they [were/were not] the most appropriate?
- (if no) Do you think there are any company personnel or areas that would
have been more appropriate for those sections?
- Who are they?
- Why would they have been more appropriate, in your opinion?
- Did you seek assistance from these people or areas?
- (if yes) In your opinion, what prevented you from getting their
assistance?
Part VII: Section 6, Intellectual Property, Technology Transfer, and Innovation
About the respondent:
- What is your role in the company? What are your responsibilities?
- Where in the company are you located?
- What is your relationship to your company’s R&D activities?
- Do you interact with managers or executives who oversee R&D activities, or their staff?
- (if yes) Please describe these interactions.
Let’s take a look at section 6. Please tell me in your own words what section 6 is about. What kinds of information does it ask for?
Now let’s look at some specific items. We’ll skip around to make sure we cover the high-priority items, then go back over some others if there is time.
Highest priority items:
Instructions
- Do you recall reading the text on page 50?
- Is there anything on that page that helped you in completing the form?
- What do you think about the section “Are responses to this survey confidential?”
6.1
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- What does [each line] mean to you?
- What does “your company” mean to you in the context of this section?
6.3
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
- Did you have to estimate to report for this item? If so, what estimation method did you use?
- In your opinion, how close is the estimated figure to an actual figure, as a
percentage?
- What does “plans to apply for” mean to you?
- What does “foreign jurisdictions” mean to you?
- Would it make any difference to you if you were asked to report a number rather than a percentage?
Lower-priority items – Probe remaining non-0 responses as time permits.
For each item:
- What does this question mean to you?
- How did you go about answering it?
- What specifically did you include in your answer?
- Did you specifically exclude anything? If so, what?
- How did you decide what to include (exclude)?
Specific probes:
6.2
- What does “apply for” mean to you?
6.4
- What does “originated within your company’s organized R&D activities” mean to you?
- Would it make any difference to you if you were asked to report a number rather than a percentage?
6.5
- What does “issued to your company by the USPTO” mean to you?
6.6
- What does “considered for patenting in 2008” mean to you?
6.7
- What does “others not owned by your company” mean to you?
- Does your company have a foreign parent?
- (if yes) Would patent licenses to affiliates of your company’s foreign
parent be included on this line?
6.9
- What does [each line] mean to you?
6.10
- What does [each line] mean to you?
- (each line) How did you decide which answer to choose?
- Would you have selected another option if one were available? Which other option would you have preferred?
- What does “important to your company” mean to you in the context of this item?
6.12
- How did you go about answering this question?
- Did your response originally include a fraction of an hour? If so, how did
you decide what to report?
- What kinds of activities did you consider?
- Does this answer include only your time, or also time spent by others?
6.13
- Are there any people who assisted with preparation of this section of the survey who are not listed here? How many? Who (titles)?
Wrap-up
- Overall, how confident are you that the information you provided matches what the survey is asking for? Which questions are you most confident about? Which questions are you least confident about?
- Overall, how easy or difficult was this section to complete? Which questions were most difficult in your opinion? Which ones were least difficult?
- In your opinion, are you the most appropriate person in the company to complete this section of the survey?
- (yes or no) What makes you [not] the best person to complete this
section?
- Did you need help from others for any parts of this section?
- (if yes) In your opinion, were you able to get help from the most appropriate people in the company?
- (yes or no) Why do you think they [were/were not] the most appropriate?
- (if no) Do you think there are any company personnel or areas that would
have been more appropriate for those sections?
- Who are they?
- Why would they have been more appropriate, in your opinion?
- Did you seek assistance from these people or areas?
- (if yes) In your opinion, what prevented you from getting their
assistance?
| File Type | application/msword |
| File Title | Business R&D and Innovation Survey – Respondent Debriefing Protocol |
| Author | tuttl004 |
| Last Modified By | Bureau Of The Census |
| File Modified | 2009-03-25 |
| File Created | 2009-03-25 |