Protocol

Cognitive Interviewing Guide: INNOVATION MODULE for ABS (3^rd Year)

Respondent’s company should have been identified as an innovator and confirmed.

Boxes indicate questions being evaluated by following probes.

Interviewer notes are in red.

Introduction:

• Explain purpose of meeting: to obtain feedback on reporting to the ABS, to understand the process of answering questions, and to evaluate how new questions work or don’t work.

• Some questions will ask may seem odd and/or obvious, but we don’t want to assume we know what you are thinking.

• Explain that we are not testing the respondent – we only want to evaluate the questionnaire

• Structure of meeting: Understand your business and your role in it, then review the questionnaire and ask you questions as you complete it.

• Permission to record discussion for note taking purposes? This study is being conducted under the authority of Title 13 USC. We plan to use your feedback to improve the design and layout of the form for future data collections. Only staff involved in this product design research will have access to the recording. Have R sign consent form.

Research Questions to Address:

• Are respondents able to answer the questions as intended?

• Are there any items that cause the respondents problems?

  • What can be done to fix those problems?

• Are there any order/context effects?

Before the Questionnaire:

• What types of goods or services does this business provide?

• What is your role in the company? What kind of responsibilities do you have?

• Are you one of the owners of the business?

• Have you completed other surveys from the Census Bureau or other agencies? If so, which ones?

• What was your role in the process for responding to this questionnaire? (Gather data? Enter data? Consult with data providers? Etc.)

REVISED INNOVATION MODULE for ABS (3^rd Year)

Provide ‘Innovation Handout 1’ to participant.

SECTION C: PRODUCTS AND BUSINESS PROCESSES

The following section collects information on the business’s introduction of a new or improved product (goods or services) or business process that differed significantly from the business’s previous products or processes.

The products (goods or services) or business processes must have characteristics or intended uses that are new or which provide a significant improvement over what was previously used or sold by the business. However, they can fail or take time to prove themselves.

The products or business process need only be new or improved for the business. It could have been originally developed or used by other businesses or organizations.

The following section asks about the three previous years including the calendar year 2018 instead of one year as in other sections of this questionnaire.

• Looking at the title and instructions, does this look like a section that you would answer?

  • If no: Who would be the most appropriate person to answer these questions?

• Based on the instructions you just read, did your company have new or improved goods or services?

  • Were these goods or services new to the firm? Were they new to the industry?

  • Please describe those goods or services.

• What does ‘differed significantly’ mean to you in this context?

• Now, we’re going to skip past some of these beginning questions in the interest of time. They may look familiar as they are already on the form, and we want to focus on the new draft questions.

Expectations

As of December 2018, did the new or improved products introduced during 2016 to 2018 meet your business’s expectations?

• Yes, expectations were exceeded

• Yes, expectations were met

• No, Expectations were not met

• Too early to tell

• How would you answer this question?

• How easy or difficult is it to recall the 2016-2018 time frame?

• What does “meet your business’s expectations” mean to you? How would you or your company determine if a product had met expectations?

  • Examples if needed: increased revenue, increased efficiency, increased brand recognition, etc…

• Would someone else in your company need to make this determination? If so, who?

• If you had more than one new or improved products or services, how would you determine your answer?

  • What if those products/services had performed differently, such as one failing and one succeeding?

• Do you have documentation to track expectations for your new or improved products/services?

  • What information is included in that documentation?

  • Is this documentation something you would have access to?

SOURCES OF New or improved goods or services

Who developed these new or improved products (goods and services)? Select all that apply

This business by itself

This business together with other businesses, universities, research institutes, nonprofits, or other organizations

This business by adapting or modifying products originally developed by other businesses or organizations

Other businesses or organizations

• How would you answer this question?

• What does ‘other businesses or organizations’ mean to you?

  • Can you give me an example?

• If respondent answers with option 2:

  • Can you describe that development process you went through?

  • What type of organization did you work with?

  • Were you the leading partner?

  • Can you describe how your organizations cooperated? How was the workload distributed?

PRODUCT COOPERATION PARTNERS

(This question should be asked only of respondents who selected option 2 or 4 of the above question.)

During the three years 2016 to 2018, did this business cooperate with any of the following partners in developing new or improved goods or services?

Yes No

1. Parent, affiliated, or subsidiary business

2. Suppliers of equipment, materials, components, or software

3. Clients or customers from the private sector

4. Clients or customers from the public sector

5. Competitors or other businesses in your industry

6. Consultants and commercial laboratories

7. Universities, colleges or other higher education institutions

8. Government, public or private research institutes

9. Non-profit organizations

10. Households or individuals

• How would you answer this question?

  • If participant answered yes to any categories: Could you give some examples?

  • If participant answered no to all categories: What kinds of businesses or organizations were you thinking of when you answered the previous question?

    • How would you describe them?

• Were there any categories that were unclear?

• Is this something you would have in your records?

Where were these businesses or organizations located? Select all that apply

[NOTE: Question below is populated only with yes responses from above]

	United States	Canada or Mexico	Rest of World
Parent, affiliated, or subsidiary business
Suppliers of equipment, materials, components, or software
Clients or customers from the private sector
Clients or customers from the public sector
Competitors or other businesses in your industry
Consultants and commercial laboratories
Universities, colleges or other higher education institutions
Government, public or private research institutes
Non-profit organizations
Households or individuals

• How would you answer this question?

• If you worked with the domestic location of a foreign-owned company, how would you report that?

• What does “rest of world” mean to you? Is there some other phrase that you think would be better?

• Is this something you would have in your records?

MOST IMPORTANT INNOVATION

Think about the most important good or service your company introduced during 2016 to 2018, was that good or service new to the market or new only to your business?

Yes No

New to the market - This business introduced a new or improved product (good or service) that was not previously offered by any of your competitors (it may have already been available in other markets)

New only to this business - This business introduced a new or improved product (good or service) that was identical or very similar to products already offered by your competitors

• How would you answer this question?

• Did your company have multiple innovations during this time frame?

• What does ‘most important’ mean to you or your company?

  • What if the question asked you to ‘think about the good or service your company introduced during 2016 to 2018 that accounts for the most revenue’?

  • Would your answer change?

• Is this something that would be formally documented somewhere?

MOST IMPORTANT INNOVATION AND SALES

What percentage of this business’s domestic sales and revenues in 2018 were attributed to your company’s most important innovation?

%

• How would you answer this question?

  • How would you calculate that answer?

• Is this something you would have in your records?

• Up to now, we’ve asked you to think about an innovation that was introduced between 2016 and 2018, but this question asks only about the revenue reported in 2018. Did you consider this when formulating your answer?

government support

During the three years 2016 to 2018, did this business use any of the following types of government programs (federal, state, or local) to aid the following innovation activities?

• Research and development

• Engineering and design activities

• Marketing and brand equity activities

• Intellectual Property (IP) related activities

• Employee training

• Software development and database activities

• Acquisition of machinery, equipment and other tangible assets

• Management related to innovation

	Yes	No
Government tax incentive or tax credit programs Include programs intended to support innovation activities such as research and development or capital expenditures.
Government grants and contributions programs Include programs intended to support innovation activities such as research and development, business development or commercialization of intellectual property.
Government training and hiring programs Include programs intended to support innovation activities such as hiring and training of researchers, interns or other personnel.
Government procurement Include programs intended to support innovation activities such as the procurement of new or improved goods, services or businesses processes.
Other government programs Include programs and activities not included elsewhere such as access to facilities, export incentives, technical assistance, market information or loans.

• How did you decide on your answers to these items?

• Did you read the descriptions of the various categories? Were they helpful or unhelpful in determining your answer?

• Who would know whether or not your company took part in any of these programs?

• How easy of difficult was it to answer this question thinking only of your company’s innovation activities?

• If yes to any: can you tell me about the process of applying for that [support]?

  • Did you have to submit a proposal? Do you know which agency you had to work with?

  • Can you describe the type/amount of support provided?

• Is this something you would have in your records?

BARRIERS and DRIVERS

• Thinking about the innovation activities we’ve been discussing, what would you say are the most important things that encourage (or allow) your company to innovate?

• What are the things that may have discouraged your company from innovating?

• Are these factors formally listed or documented anywhere?

• If we provided you with a list of encouraging and discouraging factors, would you be able to rate each one based on importance?

• Provide ‘Drivers and Barriers of Innovation Handout’ to participant.

  • Looking at this list, did any of these factors affect your company’s innovation activities?

  • Do the headings for the drivers mean anything to you?

    • What does mean ‘market factors’ mean to you?

    • How about Public policy factors? Societal factors?

BUSINESS STRATEGIES

During 2016 to 2018, to what extent did the business focus on the following business strategies:

	A great extent	To some extent	To a small extent	Not at all
Focusing on improving existing goods or services
Focusing on introducing new goods or services
Focusing on reducing prices for its goods and services
Focusing on increasing the quality of its goods and services
Focusing on broad range of products
Focusing on one or a small number of key products
Focusing on satisfying established customer groups
Focusing on reaching out to new customer groups
Focusing on standardized products
Focusing on customer-specific solutions

• Provide the ‘Business Strategies’ questions version A or B (alternating versions for each other participant)

• How would you answer this question?

• Would you need to consult with anyone else in your company to answer these questions?

• Is this something you would have documented somewhere?

• What does ‘extent’ mean to you in this context?

• What if the questions were asking you to rate the ‘importance’ of each of these focuses to your business? Would that be easier or more difficult to answer?

• What if these were phrased as ‘yes or no’ questions? For example:

  • During 2016 to 2018, did this business have a strategy in place to improve its existing goods and services? Yes or No

• Do you prefer this format of [individual questions/a table] or would you prefer this format? Why? Provide the other version for comparison.

• In general, when answering a series of questions like this, do you prefer individual questions or a table?

• Thinking about all of the questions we’ve gone over so far today, how long do you think it would take to complete these questions?

We’re interested in how companies are digitizing, or utilizing digital technologies.

• Can you tell me what digitalization means to you?

• Does your company use any digital technologies? Can you give me some examples?

• Does your company have any documentation about how these digital technologies are used?

• Does your company have any documented policies regarding digital technologies?

Now, I’d like to do an activity. I have here a stack of cards. Each card has an item related to digital technology on it. These cards are in no particular order. Please take a few minutes to read the cards and put them in groups you feel make the most sense. Please talk through your thought process as you arrange the cards. There are no right or wrong answers. (Give the participant the ‘Digital Technologies Cards.’ Note the order in which the respondent lists the cards.)

• For each group the participant creates: How would you describe this group? How would you label it?

• Do you feel like there are any technologies or concepts missing from these cards?

• Are there any cards that you feel do not belong?

• Probe on any cards participant had difficulty placing.

• Looking at the items on the cards, would you or someone in your company know which of these technologies this company used for innovation purposes?

• Is this something that would be in your records?

• Can you think of any other digital technologies your company uses for innovation activities?

• Would you or someone in your company be able to determine to what extent your company utilized these technologies for innovation activities?

• Would you or someone in your company be able to determine how important these technologies were for your company’s innovation activities?

I’m going to show you a few lists of concepts and ask you some questions about them

• Give participant the first ‘Technologies Capabilities List.’

• Take your time to read these and let me know when you’re done.

• Please describe this list of activities in your own words.

  • If needed: how would you label this list?

• Does your company engage in any of these activities?

• Would this be in your records?

• Repeat for lists two and three.

• After participant has gone over all three lists:

• Would you be able to determine the importance of each of these activities to your company’s ability to innovate separately?

• How would you define ‘technical expertise’ as it applies to your company? Can you give me examples?

• How would you define ‘design capabilities’ as it applies to you company? Can you give examples?

WRAP UP

• In general, are the innovation activities we covered today something that your company formally documents?

• Do you have any other comments of suggestions about anything we have discussed today?

THANK YOU FOR YOUR TIME!

14