Attachment Q - Crosswalk Table for Questionnaire Data Items and Research Questions

---

Overview

Cover Page
REIS QuestionJustification

---

Sheet 1: Cover Page

ATTACHMENT Q A CROSSWALK TABLE FOR RELATIONSHIPS BETWEEN REIS RESEARCH QUESTIONS OR STUTY OBJECTIVES AND EACH DATA ITEM ON ITS QUESTIONNAIRE

---

Sheet 2: REIS QuestionJustification

The Relationship between the Rural Establishment Innovation Survey (REIS) Questionnaire and Research Questions That Designed to Answer
	Question Type	Research Questions
	Innovation Processes Used to answer Research Questions 1-5	1. What percentage of rural establishments in tradable industries introduced product, process or practice innovations in the previous 3 years?
	Auxiliary Question to Differentiate Substantive from Nominal Innovators Used to answer Research Questions 2-5	2. What percentage of self-reported innovative establishments also demonstrates behaviors consistent with substantive innovation?
	Establishement Variables That May Confound Innovation Inferences Used to answer Research Questions 4-5	3. How do self-reported and ostensibly substantive innovation rates differ by urban/rural location, industry and establishment age?
	Local Context Associated With Innovation Used to answer Research Questions 4-5	4. What establishment and community characteristics are associated with self-reported and ostensibly substantive innovation?
	Busisness and Respondent Characteristics Used to breakdown overall response into meaningful groups for proposed analyses and to assess data quality	5. Do ostensibly substantive innovators demonstrate faster rates of employment growth or higher survival rates than claimed innovators and non-innovators?
Number	Question	How Question Relates to Study Objective
1	What is your job title?	This variable will be most useful in identifying possible source of bias based on respondent characteristics.
2	Approximately what year did the business at this location begin operating?	Establishment age is a critical variable for studying innovation as rates tend to be highly dependent on firm age. This data is not available in the proprietary sample frame. A measure of establishment age is available in the BLS sample.
3	What is the main product or service at this business location?	Improves data quality: the BLS and proprietary sample will have primary NAICS identified so this question will allow verifying the primary activity.
4	Does this business have only one location or more than one location?	Essential information related to establishment autonomy
4a	(If more than one location) Is this location the business’s headquarters or is it a branch location?	Essential information related to establishment autonomy
5	How important is each of these factors for locating this business in this community? a. Owner-ties to area b. Availability of low-cost labor c. Available skilled labor pool d. Access to transportation e. Access to broadband or high speed internet f. Access to material inputs g. Access to customers h. Government incentives i. Low taxes j. Strong or growing local economy	Adapted from the 1996 ERS Rural Manufacturing Survey, the question informs those local attributes associated with innovative firms.
5a	How important is each of these factors for making this community an attractive place to work? a. Opportunities for outdoor recreation b. Scenic beauty (e.g., natural or architectural) 5 c. Climate d. Access to arts and entertainment e. Quality of local schools f. Access to health care .	Also adapted from the 1996 ERS Rural Manufacturing Survey, the question proved to be critically important in understanding how rural firms overcame location disadvantages especially with respect to hiring highly qualified employees.
6	What was the average number of employees on your payroll in 2012, including all full-time and part-time workers at this location?	Establishment size critical control variable. From sample frame only know establishment size class (small 5-19; medium 20-99 or large 100+).
6a	Were there workers at this business, NOT ON YOUR PAYROLL in 2012, such as independent contractors, or temporary workers?	Emerging human resource practices may dramatically affect the size of the establishment when compared with convention payroll measures of employment. Greater reliance on more contingent workers may also affect innovation processes.
7	During the past 12 months, did this business…a. Offer a health insurance option for any employees .. b. Offer a retirement plan c. Pay for employee education, professional development or train d. Offer paid maternity, paternity, or family leave .. e. Have an employee-ownership plan.. f. Offer paid time off for employees to volunteer	Question elicits further information on the level of commitment in employment relationship that may affect establishment innovation
8a	How many employees are managers at this location?	First component of typical salaried workforce. Will be used to compare occupational structure of innovative and non-innovative firms.
8b	How many employees are professionals?	Second component of typical salaried workforce. Will be used to compare occupational structure of innovative and non-innovative firms.
9	For 2012, what is your best estimate of the average hourly wage for non-salaried workers at this location?	Deriving wage rates for non-salaried workers is a challenge with this survey relative to manufacturing surveys as there is no clear distinction between production and non-production workers. Our strategy is to elicit information on salaried workers in 8a and 8b and then ask about wage rates of non-salaried workers. This series of questions worked well in the cognitive interviews among manufacturing and non-manufacturing firms. Wages are an important variable for assessing employer commitment and will eventually be compared with local area wage rates from secondary data.
10	In 2012, at this location, what percent of employees were in the following occupational categories? a. Management and professional .... b. Services . c. Sales and office support .... d. Natural resources, construction, and maintenance e. Production, transportation, and material moving	The OMB summary occupation groups are used as an aid for categorizing the different types of workers at an establishment. This provides a mental aid in arriving at minimum educational requirements for different types of workers in the following question. A simpler categorization of salaried and non-salaried workers was rejected as this might blur important distinctions between, say, office support and production workers with regard to literacy or numeracy.
10a	For each occupational category what was the minimum educational level needed in 2012? < HS At least HS At least Assoc Degree/Voc Cert At least 4-year college More than 4-year college	In addition to being a confounding factor in understanding innovation processes, information from the question will provide critical information on the importance of intermediate skills (e.g., the need for a vocational certificate) that has been identified as a critical research area with no reliable secondary data.
11	Is any part of the workforce unionized or covered by a collective bargaining agreement?	A potential confounding factor, not collecting this information would make subsequent analyses vulnerable to omitted variable bias criticism.
12	In the last 3 years, how difficult has it been to find qualified applicants for your workforce, not including managers and professionals?	Difficulty in finding skilled workers is assumed to be a major impediment to innovative establishments in rural areas. This series of questions will help quantify the degree to which common knowledge comports with objective data.
12a	(If very or somewhat difficult) Why has it been difficult to find qualified applicants for your workforce? Is it…a. Quality of the labor pool b. Increases in required skills and knowledge c. Insufficient number of workers available locally d. Limited interest among job seekers for openings at this business	To derive information regarding the perceived cause of the labor shortage.
13	Does this business have written position descriptions?	This question informs the data driven decision making construct that will be used to try to differentiate substantive innovators from nominal innovators. A series of papers by Erik Brnjolfson demonstrates that data driven decision-making firms tend to be more productive and more innovative than firms that are not as reliant on data for decision-making. This human resource question was chosen because firms pursuing ISO 9000 quality control certification are most likely to be noncompliant with respect to their policies for defining positions, their training requirements and tracking whether training requirements are met. Among compliant firms, this was the one element that was deemed most critical for improving the firms' continuous improvement processes. For this reason, this is an excellent set of questions for identifying establishments that exploit information for decision-making.
13a	Are training requirements documented in those position descriptions?
13b	Does this business track whether employees complete or if they have already completed these training requirements?
14	Are the following technologies currently used at this business? a. Personal computers/laptops, not including smartphones b. Broadband or high speed internet c. Sale of products or services over the internet (e-commerce) d. Supplies purchased over the internet (e-procurement) e. Web advertising f. Direct e-mail marketing g. Social media (e.g., LinkedIn or Facebook) h. Business issued smartphones to employees i. RFID readers, barcode or optical scanners (e.g., Radio Frequency Identification) j. Computer software specifically designed for your business or industry k. An integrated enterprise resource planning system (e.g., SAP or Microsoft Dynamics, or Oracle Applications that include accounting, logistics, human resources, sales management, along with other functions) l. Stand alone supply chain, logistics management software m. Stand alone customer relationship management software	The technologies that establishment use to collect, organize and analyze data will also directly inform the data driven decision-making construct.
15	What percent of the sale of products or services comes from the internet?	The main purpose of this question is to identify virtual establishments that may differ from brick and mortar establishments with respect to innovation processes.
16	What percent of your workforce, not including managers and professionals, uses computers on a daily basis?	This question assesses the diffusion of digital technologies throughout the establishment. The assumption is that greater diffusion throughout the establishment will increase capabilities for collecting and organizing data.
17	Which of the following factors have limited this business’s use of information and communications technology? a. The cost of equipment and software b. The cost of information and communications services c. Lack of access to adequate broadband or high speed internet d. Lack of knowledge e. Difficulty integrating new technologies into the current way you do business	This question assesses local and organizational constraints to becoming a more data driven business.
18	Businesses obtain information about new opportunities or new ways of doing things from many sources. Which sources have been most valuable for this firm? a. Suppliers b. Customers c. Other business people in your industry d. Other business people NOT in your industry e. Business/trade association conferences or publications f. Your own employees g. Media (e.g., newspapers, television, internet) h. Private consultants i. University extension, community colleges, or business schools	A central conjecture as to why innovation is impeded in rural establishments is that their networks for obtaining information are much thinner than their urban peers. This question will allow comparing sources of information used by rural innovators and non-innovators and across urban and rural peers.
19	Of the sources identified as very valuable, where are they located? In your community Outside your community within a reasonable drive Beyond a reasonable drive	By identifying the location of principle sources of information, the question will allow assessing the extent to which rural innovators are able to mitigate the disadvantages of remoteness.
20	Approximately what percent of 2012 final shipments or billed services went to customers…a. Locally, within a reasonable drive b. Beyond a reasonable drive in the United States c. Internationally	The extent of the market (local, regional or international) is one indicator of the level of competition and thus the demand for innovation.
21	Was this location in business in 2007?	To produce skip for establishments founded after 2007.
21a	What is your best guess of the percent of 2007 final shipments or billed services that went to customers a. Locally, within a reasonable drive b. Beyond a reasonable drive in the United States c. Internationally	A marketing innovation could be indicated by a substantial expansion of the market area served.
22	In 2012, what percent of goods and services sold by this business were sold to…a. Other businesses b. Government c. Individuals	The dominant customer base is likely to affect innovation processes within the establishment. Failing to collect this information could contribute to missing variable bias.
23	Is the current market for your products or services…Growing Stable Declining Mixed (i.e., some are declining, others growing) Uncertain	The subjective view of industry prospects may affect innovation processes within the establishment. Failing to collect this information could contribute to missing variable bias.
24	Does this business require employees to document good work practices and lessons learned?	Responses to this question will be used to identify data driven decision-making establishments in combination with Q26-28 and Q31. Establishments responding affirmatively recognize the importance of data to institutional memory and learning .
25	How often does this business monitor customer satisfaction through analysis of complaints, customer satisfaction surveys, focus groups, or other methods? Never Occasionally Regularly	This question informs the data driven decision-making construct by assessing how proactive the establishment is in collecting and analyzing customer data.
26	How often are processes changed to fix problems indentified through customer complaints? Never Occasionally Regularly	This question informs the data driven decision-making construct by assessing the extent to which the analysis of data results in concrete action.
27	In the last 3 years did this business… a. Produce any new or significantly improved goods b. Provide any new or significantly improved services c. Introduce new or significantly improved methods of manufacturing or producing goods or services d. Introduce new or significantly improved logistics, delivery, or distribution methods for your inputs, goods, or services e. Introduce new or significantly improved support activities for your processes f. Introduce new or significant improvements in your marketing methods	This is the central self-reported innovation question derived from the Community Innovation Survey and used in BRDIS. In combination with question 29 it will allow classifying establishments as innovators or non-innovators.
28	In the last 3 years, did this business have any improvement or innovation activities that were…a. Abandoned b. Incomplete	This question serves two purposes: first to identify early stage innovators who are engaged in innovation but have not yet introduced a successful innovation and to serve as auxiliary information to differentiate substantive innovators from nominal innovators. A respondent who answered affirmatively to any of the innovation items in Q27 influenced by social desirability is much less likely to acknowledge abandoned or incomplete innovation initiatives. In contrast, substantive innovators are much more likely to answer this question affirmatively.
29	Did you check “Yes” to any answers in question 27 or question 28?	Skip for non-innovators.
29a	Please tell us why improvement or innovation activities have not been necessary or possible.	Skip for non-innovators.
30	In 2012, did this business sell any new or significantly improved goods or services with the following improvements? a. Improved performance b.More user-friendly c. Reduced costs .... d. New features e. New service capabilities	Derived from Community Innovation Survey questions to assess the nature of product innovations.
31	In the last 3 years, did this business start selling any new or significantly improved goods or services before your competitors in at least one of your markets?	Derived from Community Innovation Survey to assess novelty of innovations.
32	In 2012, what percent of this business’s sales came from new or significantly improved goods or services?	Derived from Community Innovation Survey to assess the importance of product innovation to establishment sales.
33	In the last 3 years, did this business engage in any of the following innovation-related activities? a. In-house research and development (R&D) to increase knowledge or devise innovations b. Purchase research and development (R&D) from research organizations or other branches of this business c. Conduct in-house design activities to improve aesthetics of product or packaging d. Purchase design services e. Purchase machinery, equipment, computers or software to implement innovations f. Purchase or license patents or inventions to implement innovations g. Purchase knowledge or expertise to implement innovations h. Plan, engineer, design, or conduct other development work to implement innovations i. Train staff to develop or introduce innovations j. Market research, advertising, and other marketing activities linked to implementing innovations	Derived from Community Innovation Survey to identify specific innovation related activities or innovation inputs. This is an important complement to the BRDIS survey as it will allow comparing the prevalence of some of the hard inputs to science and engineering based innovation (e.g., R&D, licensing patents, etc.) in rural and urban establishments. An important research question is the role these inputs play in substantive innovation and whether there is a difference in their relative importance for rural and urban establishments.
34	In the current environment, if excess cash were available, how likely is it that these funds would be used to…a. Provide additional training of employees b. Repay debt c. Provide a reserve or cushion d. Fund additional innovation projects e. Fund additional investment projects, such as replacing old equipment or for expansion	One of the principal auxiliary questions to try to differentiate nominal innovators from substantive innovators, the purpose of the question is to identify firms that are capital constrained with respect to innovation projects. In cognitive testing the question was easily understood and answered by respondents. Our main concern is that respondents from very large establishments may have a harder time answering the question. This will be assessed during the pilot study.
35	Does this business produce products or provide services in any of the five green sectors? a. Produce renewable energy b. Increase energy efficiency c. Conserve natural resources d. Prevent, reduce, or clean up pollution e. Produce clean transportation fuels	The green sector classification used was developed by BLS. Although the BLS green jobs data collection effort has been cancelled this issue is critically important for understanding potential differences between innovation in urban and rural areas. Since many forms of renewable energy are space intensive given direct or indirect reliance on solar power, rural establishments may have a comparative advantage in these sectors. A study of rural innovation that did not explicitly investigate an area of potential rural advantage would be incomplete.
36	In the last 3 years did this business participate in any patent applications?	Patents are the most common formal measure of innovation. Collecting information on patent applications will allow comparing formal innovation potential with the less formal substantive innovation which is the central focus of the survey. Two critical comparisons will be rural and urban patent application rates and rural and urban substantive innovation rates.
36a	In the last 3 years how many patent applications did this business participate in?	Formal measure of innovation potential prolificness will allow investigating establishment and community characteristics associated with higher innovation rates.
36b	In the last 3 years how many patent applications were successful?	Formal measure of successful innovation prolificness will allow investigating establishment and community characteristics associated with higher innovation rates.
37	In the last 3 years did this business… a. Register an industrial design b. Register a trademark c. Produce materials eligible for copyright d. Use trade secret protections (e.g. non-disclosure agreements, non-compete clauses or sought remedies for misappropriation)	Whether establishments have intellectual property worth protecting provides another dimension for differentiating substantive innovators from nominal innovators.
38	During the economic recession (2008-2009) period, to what extent did this business commit resources to innovate? 1 Increased resources for innovation activities 2 There was no change in innovation resources 3 Delayed or decreased resources for innovation 4 Not applicable, e.g. not in business at that time	The impact of the Great Recession on innovation initiatives is a very rich empirical question with the potential for broad implications for government policy. The relative closeness of the Great Recession to the study period provides a unique opportunity for investigating this question.
39	Compared to 2012, in this current year (2013) would you say resources for innovation at this business have been… 1 Increased 2 Kept the same 3 Decreased	The current trend in innovation funding during a period of slow but steady recovery provides useful context for interpreting responses to Q38.
40	Over the last three years has this business…a. Increased variety of goods or services offered b. Increased market share or entered new markets c. Begun exporting goods or services d. Reduced time to respond to customer needs e. Improved flexibility of production or service provision f. Increased capacity of production or service provision g. Reduced labor costs per unit output h. Reduced materials and energy required per unit output i. Improved employee satisfaction/reduced worker turnover	This question assesses the outcomes in terms of establishment performance. Since this question is asked of all establishments, it will allow comparing the outcomes of self-reported innovating and non-innovating firms, as well as the outcomes of substantive and nominal innovation establishments. Non-innovating establishments with broad performance improvements will be of particular interest in assessing the self-reported innovation measures. For instance, if non-innovators with broad performance improvements also tend to respond to the auxiliary questions similar to substantive innovators this would suggest 4 subpopulations: substantive innovators, nominal innovators, non-innovators and self-reported non-innovators with effective continuous improvement processes.
41	The following is a list of factors related to this business’s location. For each one please tell us how much of a problem it is for this business’s ability to compete. a. Zoning or development regulations b. Vitality of local economy c. Access to financial, legal, and other business services d. Access to equipment and software suppliers e. Access to training courses f. Access to transportation/freight forwarding facilities and services g. Availability of broadband or high speed internet h. Local availability of mobile or cellular service i. Local roads and bridges j. Cost of facilities and land k. Attractiveness of area to managers and professionals l. Quality of primary and secondary schools m. Environmental regulations n. State and local tax rate	This question, derived from the 1996 ERS Rural Manufacturing Survey was initially used to assess how local circumstances affected business competitiveness. For the purposes of the present study the question will inform how local circumstances affect establishment innovation.
42	How involved in promoting business are the following institutions in your community? a. Local government development effort 1. Village, town or city 2. County 3. Regional or multi-county b. Business association (e.g., Chamber of Commerce) c. College, university or extension support for local business d. Community foundations or nonprofit organizations e. Local investors f. Banks	The question will allow assessing the role that social capital as related to business plays in innovation.
42a	How much civic leadership does THIS business provide in the community?	This question allows addressing the reciprocal of the prior question: how are innovation processes in the establishment associated with the provision of social capital.
43	Do you have a good understanding of the decisions that led to the founding of this business?	Q43-43b collect information on user entrepreneurship within the establishment. User entrepreneurship is when a business is developed around a good service developed first for the entrepreneurs own use. Kauffman Foundation research has confirmed that user entrepreneurs are more likely to be funded with venture capital and to have formal R&D activities than non-user entrepreneur peer firms.
43a	(If Yes) Was the business originally founded around a new or customized product or service that was created by one of the founders of the business?	There is anecdotal evidence that user entrepreneurship is common in rural areas given relatively thin markets and more limited information on existing products that might fill a user need.
43b	(If Yes) Thinking about this new or customized product or service, why was it originally developed? 1 One of the founders created it for personal use 2 One of the founders created it for use at a previous job or business 3 One of the founders identified a business opportunity	The central questions to be addressed are whether user entrepreneurship is more prevalent in rural areas than urban areas and whether user entrepreneurship in rural areas enjoys the same advantages of venture funding and R&D activity that characterize user entrepreneurs nationally.
44	The final section of the survey is to assess which types of government or government-sponsored programs are the most helpful to businesses. If you used, please rate how important each program has been for this business in the last 3 years. a. Direct loans from a government agency (e.g., USDA B&I Direct Loan) b. Government insurance or guarantee for loans (e.g., SBA 7(a) Loan Program) c. Revolving loan funds run by a nonprofit or government organization d. Tax incentives by state and/or local government, including enterprise zones, urban revitalization areas, Tax Increment Financing districts e. Government-assisted industrial parks or business incubators f. Government-funded technology assistance programs g. Government-assisted worker-training programs h. National Innovation Marketplace	This question provides information on the use of government sponsored programs mainly related to finance but also including technical assistance and training. A variant of this question was used in the 1996 ERS Rural Manufacturing Survey and received few responses. Responses to this question will be closely tracked in the pilot to determine if it warrants inclusion in the full study.
45	Did this business try to borrow money for any purpose over the past three years?	Skip for debt-free establishments which our cognitive testing suggested are common.
45a	Please indicate the sources you tried to borrow from below. If you did try to borrow money from a source, how much of the funding did you receive? a. Commercial bank b. Savings & Loan or credit union c. Finance or leasing company d. Insurance or mortgage company e. Family or friends f. Federal, state or local government g. Credit or advance from a customer h. Angel capital funding i. Venture capital funding j Personal sources of funds (other than credit cards) k. Personal home equity loan l. Other personal loan m. Personal credit card	The environment for debt financing has been very tough for many businesses since 2008. Thus, the question asked both whether the establishment tried to borrow from a particular, and if so, how much of the funding request was received. Again, a variant of this question was used in the 1996 ERS Rural Manufacturing Survey. Responses were more numerous relative to the government sponsored programs question but these data did not provide any additional insight into business competitiveness. Dropping the question is less likely given the potential for omitted variable bias. If nothing else, it will probably be necessary to distinguish debt-free establishments from establishments that borrowed funds.
46	How were the funds this business borrowed or wanted to borrow to be used? a. Cash flow or operating costs b. Real estate or structures c. Replacement of old industrial plant location, equipment or vehicles d. Investment in additional plant, equipment or vehicles e. Repayment of debt f. Reserve or cushion g. Inventory h. Fund innovation projects i. Investment in intangible assets such as branding, training, or design	In concert with the hypothetical Q34, this question will help differentiate substantive innovators from nominal innovators based on responses to 46h and possibly 46i
47	Over the past 3 years, were business profits (retained earnings) used to finance the business?	Skip if retained earning not used to finance business.
47a	Compared with borrowed funds, how important were business profits for funding investment? 1 More important 2 Less important 3 Equally important 4 Don’t know/Not applicable	The importance of retained earnings relative to external sources of finance may be associated with different innovation processes, and this may vary by size of establishment. For example, small establishments more dependent on retained earning may have fewer constraints on innovation activities whereas large establishments more dependent on retained earnings may be capital constrained. Combining information from Q34 will provide useful insights.
48	Which of the following best describes your current position? 1 Mid level manager 2 Senior manager 3 Executive/owner 4 Other, please describe	Respondent characteristic that may be associated with item nonresponse or bias.
49	Which statement best describes your familiarity with how innovation is carried out in this business? 1 Not familiar 2 Slightly familiar 3 Moderately familiar 4 Strongly familiar 5 Completely familiar	Respondent characteristic that may be associated with item nonresponse or bias.
50	What is your gender?	Respondent characteristic that may be associated with item nonresponse or bias.
51	How long have you been employed at this business?	Respondent characteristic that may be associated with item nonresponse or bias.
52	That is my last question. If you have any additional comments about this survey or innovation in general, please write them in the box below.	Open ended question to pick up newly emerging issues related to business innovation.