Section_A_Supporting Statement 9-18-09

SUPPORTING STATEMENT

U. S. Department of Energy

Profiles of Innovativeness and Effective Research Communication

OMB Control Number: (NEW)

A. Justification

1. Explain the circumstances that make this collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The National Science Foundation (hereafter NSF), has established a new funding program on the Science of Science and Innovation Policy (SciSIP) in order to develop a scientific approach to the formulation of science and innovation policy. This proposal is one of the first to be funded under this initiative. In addition, this project is an extension of a long-term research effort funded by the Department of Energy’s (DOE) Office of Basic Energy Sciences (BES) to develop best practices in the management of scientific innovation and supports the DOE Undersecretary for Science in advising the Secretary of Energy with respect to: 1) the well-being and management of the multipurpose laboratories under the jurisdiction of the Department; and 2) the long-term planning, coordination, and development of a strategic framework for Department research and development activities, as required under, The Energy Policy Act of 2005, codified at 42 U.S.C. §§ 7132(b)(4)(C) and (F). Recently Dr. Orszag has called for the importance of identifying those programs that work and those that do not, and this research study is designed to help begin to answer this question for one of the most important of all objectives, scientific innovations.

Our objective is to develop explanatory hypotheses about the characteristics of research projects that facilitate innovation in a qualitative study that then could be tested in more rigorous research. For example, one potential hypothesis is whether cross-disciplinary communication increases the rate of scientific innovation. Studying these types of questions is complex, because research projects differ in important ways:

1. Relative emphasis on high risk research;

2. Relative scope of the problem and size of the project

3. The nature of the research area: biology, chemistry, alternative energy, etc.

Characteristics of projects that facilitate innovation in large projects might not be the same as those in small ones. Further, what is beneficial for biological research projects might not have the same effect for chemical sciences research projects. In so far as patterns are established in this qualitative study, they can undergo testing in a much larger and more expensive study. Such a followup study could be used to establish best practices for promoting innovation in different kinds of research projects. Of course, any such follow-up study would need to be resubmitted to OMB for review and approval.

A novel feature of this qualitative study is that it allows the staff responding to indicate certain practices that they would like to see emphasized by research managers. This differs from current case studies that ask managers, not staff, for opinions on what leads to innovation. Particularly looking across the various cases, this can provide a considerable amount of useful feedback to directors of the national laboratories, as well as the Office of Basic Energy Sciences in the DOE. Furthermore, these multiple case studies will provide legitimacy to managers as they attempt to facilitate and foster innovative research to support their missions.

Since the interviews involve human subjects, the interview guides utilized in this study will be reviewed by the University of Maryland’s Institution Review Board and the DOE Laboratories’ Human Subjects Boards in order to ensure compliance with the requirements on biomedical and behavioral research involving human subjects set forth in the National Research Act (P.L. 93-348) and the regulations on public welfare set forth in Part 46 of Title 45 of the Code of Federal Regulations (45 CFR 46). Participation will be voluntary and the individual data collected will be protected from use for anything other than this and related research. All data analysis will be by specific kinds of research projects (large vs. small or alternative energy vs. material sciences, etc.) so that the attitudes of individuals cannot be identified. The Center for Innovation at the University of Maryland is the recipient of the DOE Basic Energy Services funding through Sandia National Laboratories and a NSF grant and is responsible for the collection and analysis of the data and will protect confidentiality.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

How is the data collected?

Four sets of data are to be collected. The most important is a survey (A-2) developed with funding by the DOE Office of Basic Energy Sciences that will be given to all the members of 72 research projects in six national laboratories. This survey contains questions about a number of attributes related to potential best practices. The projects will be selected by middle managers based on criteria that reflect four kinds of research projects large and small projects with both incremental and radical innovation objectives. The projects will be located in six national laboratories so that it is possible to select projects in each of the following research areas:

1. Biological sciences;

2. Chemical sciences;

3. Alternative energy;

4. Material sciences;

5. Geophysical sciences.

The projects selected represent a purposive, non-representative sample, and no inferences can be made to the general population of research projects in national laboratories.

In addition to administering the survey to all members of the selected projects, we will conduct semi-structured interviews with three distinct groups of organizational participants. First, we will interview middle managers of the selected group of projects (see A-1). The interview will encompass such things as the disciplinary context and the presence of various kinds of policies to encourage the development of complex research teams and cross-fertilization of ideas, two best practices identified in the management of innovation literature. Since one of the major concerns is the extent of scientific innovation, the project leaders of the selected 72 projects will be interviewed (A-3) at the beginning and the end of the project about the extent of scientific innovation. Finally, top managers in each national laboratory (A-4) will be interviewed about strategies for building diverse work teams and encouraging the exchange of information.

By whom is the data collected?

The National Science Foundation has awarded a grant to collect this data to the Center for Innovation at the University of Maryland. In addition, the Department of Energy’s Office of Basic Energy Sciences, through Sandia National Laboratories, has been funding the Center for Innovation to develop understanding of the key elements in research environments that contribute to the ability of staff to accomplish excellent research and developing tools for addressing and improving such environments.

For What Purpose?

Although this is an exploratory, qualitative study, we believe that the information gathered serves four distinct purposes.

First, the analysis will help in the construction of a science of science and innovation policy and help meet the objectives of management and the heads of national research laboratories, including leadership of the Department of Energy. This constituency will receive reports on the patterns of findings and may decide to implement policies or programs on an experimental basis based on some findings. In this case, we would seek to remain in longer-term contact in order to facilitate more robust collection of the data. For example, there is increasing evidence that the diversity of the research team appears to stimulate innovation if there is cross-fertilization and/or external collaborations. If we find similar patterns across both small and large projects, those with incremental and high risk research, and in all five disciplinary contexts, then managers might want to explore ways of increasing the diversity of research teams. Furthermore, since part of the study attempts to identify practices that encourage cross-fertilization and external collaborations, insights about this might prove helpful to senior managers who are interested in encouraging these types of activities.

Second, middle managers and project leaders who participate in the study will be given reports with summary data on all 72 projects and the similarities and differences between the projects in their national laboratory and those in other national laboratories, without designating laboratories by name. These kinds of comparative reports can provide useful insights and might lead them to change their current managerial practices in the light of what appears to be stimulating more innovation elsewhere.

Third, since survey respondents will be able to express their desires for change, a summary of these responses will be reported back to middle managers and project leaders as well as senior management of the national laboratories that are involved. As we have suggested, for these projects and projects similar to them, this multiple case study provides not only useful feedback but also legitimacy for organizational changes designed to promote innovation. This kind of qualitative information becomes more compelling if the researchers in one laboratory ask for more critical feedback as a best practice and in other national laboratory projects this appears to be associated with more innovation.

Fourth, academic papers will be prepared for diverse audiences including those who read science policy, evaluation, and management, as well as organizational sociology and business management journals. Thus, the data collection, even though it is a qualitative study, meets a wide range of objectives of multiple audiences.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

At this time, it is anticipated that the survey data will be collected via website to reduce the cost of the survey and to facilitate the efficiency of the data collection. We anticipate that the survey website and database will be maintained by a DOE-approved third-party vendor that has conducted similar surveys for the DOE in the past. To provide site security, the computer system will employ security software programs to monitor network traffic to identify unauthorized attempts to upload or change information or otherwise cause damage. Unauthorized attempts to upload information or change information on the website would be punishable under the Computer Fraud and Abuse Act of 1986 and the National Information Infrastructure Protection Act.

4. Describe effort to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in item 2 above.

The survey that will be used has been specifically designed to measure best practices in the promoting innovative research. Furthermore, it goes considerably beyond the present management of innovation literature in identifying potential levers and mechanisms for best practices. To date, there has been an absence of systematic data collection and analysis at the project level, especially in the national laboratories. Literature and web reviews have revealed no comparable survey instruments.

5. If the collection of information impacts small business or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize burden.

No small businesses are involved in this data collection.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

Absence of information about environments that encourage innovation means that we do not have all the needed tools for facilitating and fostering excellent research. In particular, there has been a lack of attention to how best practices vary by the type of research project, as well as a general tendency to think that all research projects can and should be treated in the same way. This project will seek to locate systematic differences among research projects and best practices. Although it is a qualitative study, as we have indicated above, it can inform present management practices.

Furthermore, without this data one cannot begin to explore whether the ideas developed and demonstrated in the industrial innovation literature can be applied to scientific innovation. In particular, the social science literature has not explored how organizational policies impact on the creation of complex research teams and the cross-fertilization of ideas, two factors associated with better performance in the industrial innovation literature. With the growth in knowledge in recent decades, the arrangement of research in the larger system of knowledge production has changed dramatically. Part of the data collection effort is determining whether policies are currently in place that might inhibit the United States from capitalizing more effectively on its investments in scientific research. DOE and the NSF have indicated that this information might have enormous potential for informing government policy about how to improve the pursuit of scientific innovation.

Finally, the absence of this data does not allow managers of national laboratories to have objective data to assess the impact of changes in organizational processes and structures. In particular, we are interested in providing a basis to assess if there is an impact on a range of strategies and policies, such as those that affect the formation of complex research teams within and between the national laboratories and the facilitation of the cross-fertilization of ideas.

7. Explain any special circumstances that require the collection to be conducted in a manner:

• requiring respondents to report information to the agency more often than quarterly;

• requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• requiring respondents to submit more than an original and two copies of any document;

• requiring respondents to retain records, other than health, medical, government contract, grain-in-aid, or tax records for more than three years;

• in connection with a statistical survey, that is not designed to produce valid, reliable results than can be generalized to the universe of the study;

• requiring the use of statistical data classification that has not been reviewed and approved by OMB;

• that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.

The data collection discussed in this clearance will be conducted in accordance with the guidelines in 5 CFR 1320.5. The data collection for which approval is sought:

• is the least burdensome necessary for the proper performance of the DOE’s function to comply with legal requirements and achieve program objectives;

• is not duplicative of information otherwise accessible to the DOE; and

• has practical utility.

In addition, the data collection for which approval is sought under this clearance request:

• does not require respondents to report information to the agency more often than quarterly;

• does not require respondents to prepare a written response to a collection of information in fewer than 30 days after the receipt of it, with the exception of a telephone or a web-based survey which would result in voluntary responses in less than 30 days from receipt or contact;

• does not require respondents to submit more than an original of any document;

• does not require respondents to retain records for more than three years;

• is not designed to produce results that can be generalized to the universe of study, defined as large public research laboratories;

• does not require the use of a statistical data classification that has not been reviewed and approved by OMB;

• does not include a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• does not require respondents to submit proprietary or confidential information.

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency’s notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and record keeping, disclosure, or reporting format (if any), and on the data element to be recorded, disclosed, or reported.

Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every 3 years – even if the collection of information activity is the same as in the prior periods. There may be circumstances that mitigate against consultation in a specific situation. These circumstances should be explained.

The Federal Register notice was published in Volume 73, No. 196, Wednesday, October 8, 2008 on page 58945.

Public comments received:

1) from Ms. Cheryl A. Fragiadakis, Department Head, Technology Transfer and Intellectual Property Management Lawrence Berkeley National Laboratory, asking if DOE is in communication with the Industrial Research Institute (IRI), which is engaged in "research on research", primarily in private sector research organizations.

Response (summary): We were familiar with the IRI’s Research on Research (ROR) Committee through Sandia’s membership in IRI, visits with committee members about our research related to this effort, and having published in and read the IRI journal. We also made certain we were current by contacting a former ROR Chairman, Parry Norling, who inquired for us at IRI. We concluded that our study is not duplicative of efforts at IRI. Two current research projects being undertaken at IRI have different research scope, questions, and design and thus do not coincide with this study. These are Leveraging Thought Diversity, which is exploring how thought diversity among team members affects team dynamics and innovation outcomes, and Collaboration Continuum, which is looking at knowledge collaboration and its role in driving productivity and innovation. It is focused on analyzing social networking and other Enterprise 2.0 software.

2) from Mr. Daniel Berkovits, Policy/Research Assistant, Metropolitan Policy, Program, The Brookings Institution, Washington, DC 20036 requesting all relevant instruments and supporting documentation.

Response: Provision of the instruments and documentation was delayed pending resolution of the copyright of the survey instrument.

Resolution: There is now no copyright and a copy of the instruments and supporting documentation was sent to Mr. Berkovits.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

Neither Office of Basic Energy Sciences nor the Center for Innovation at the University of Maryland, which is responsible for the data collection, will provide any payments.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

The survey and structured interviews will indicate that efforts will be made to protect confidentiality through the following procedures:

1. All hard copy and electronic data will be securely stored at the Center for Innovation to prevent unauthorized access, disclosure, or loss. Hard copy records will be stored in locked filing cabinets which only the principal investigators can access. Electronic data will be stored on an external hard drive connected to the computer of the PI (Hage). This computer has appropriate security safeguards, including unique identification of authorized users, password protection, automated operating system patch (bug fix) management, anti-virus controls, firewall configuration, and scheduled and automatic backups to protect against data loss or theft.

2. Upon data entry, the electronic survey responses will be purged of any connection to personal emails and/or any other identifying information. Open-ended comments will be edited to remove any occurrences where employees have signed or otherwise indicated their identity in the survey. Hard or taped copy of interviews and surveys, if any, will be destroyed upon conclusion of the study.

3. Survey results will be publicly reported to respondents and managers. Prior to reporting results, all output will be reviewed to ensure that it does not reveal the identity of any single individual.

3. All survey respondents and interviewees will receive information about these efforts to protect confidentiality, and will indicate their voluntary consent to participate prior to commencing the activity.

3. An expert in disclosure avoidance will be identified and retained to assess the abovementioned procedures and monitor adherence. The selection of the expert will be reported to OMB within six months.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. The justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation given to person from whom the information is requested, and any steps taken to obtain their consent.

No sensitive data is collected.

12. Provide estimates of the hour burden of the collection of information. The statement should:

• Indicate the number of respondents, frequency of response, annual hour burden, and explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If hour burden on respondents is expected to vary widely because of differences in activity, size or complexity, show the range of estimated hour burden and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.

• If request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.

• Provide estimate of annualized cost to respondents for the hour burdens of collections of information, identifying and using appropriate wage categories. The cost of contracting out or paying outside parties for information collection should not be included here. Instead, these costs should be included in Item 13.

	FY09			FY10
	Respondents	Hours	Cost	Respondents	Hours	Cost
Survey (A-2)	504	252	$13,310	504	252	$13,310
Middle Managers (A-1)	21	21	$1,109	21	21	$1,109
Project Leaders (A-3)	36	36	$1,902	36	36	$1,902
Top Managers (A-4)	9	9	$475	9	9	$475

There are no known out-of-pocket costs to the respondents. The labor cost associated with completing the survey is indicated in Attachment A as well as below and is based on an average hourly wage of $52.82 (based on the mean wage for the BLS occupational category of engineering manager in the 2006 National Compensation Survey). This figure likely does not include the overhead costs of the national laboratories.

The calculation of the number of respondents is based on the following assumptions. We assume that 16 projects will be selected from each of the three large laboratories (Brookhaven, Pacific Northwest and Sandia) and 8 projects will be selected from each of the three small laboratories (Ames, National Renewable Energy Laboratory, and NOAA). We will attempt to select projects such that there are equal numbers of small and large project in each laboratory. In the large laboratories, we assume that the small projects will each have approximately 8 researchers, while the large projects will each have approximately 25 researchers. In the smaller laboratories, we are assuming that the small projects will each have 8 researchers, while the 4 large projects will have approximately 10 researchers. All researchers from the selected projects will receive the survey (A-2). Three middle managers (A-1) in each department and center will be interviewed to select the research projects or a total of 42, to describe the characteristics of their research area and to indicate policies that facilitate the creation of complex research teams and cross-fertilization. Three top managers in each national laboratory will receive a structured interview (A-4) about strategies to develop collaborations between departments within the laboratory and with other research organizations, for a total of 18. All seventy-two project leaders will undertake a semi-structured interview (A-3) about scientific innovation in their projects.

Although surveys and interviews are displayed as distributed equally between FY2009 and FY2010, the actual distribution might different after the research has begun and we negotiate access to the laboratories.

13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden show in Items 12 and 14)

The majority of activities will be undertaken by the Center for Innovation at the University of Maryland. See Attachment A for a proposed budget that encompasses the annualized costs for the Center’s activities which are funded in part by NSF and in part by Basic Energy Sciences of DOE.

14. Provide estimates of annualized costs to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing and support staff), and any other expenses that would not have been incurred without this collection of information. Agencies may also aggregate cost estimates from paragraphs 12, 13 and 14 in a single table.

No annualized costs for the government exist because this data collection effort is being funded as a research activity by NSF and the Office of Basic Energy Sciences.

15. Explain the reasons for any program changes or adjustments in Items 13 or 14 of the OMB Form 83-I.

This is a new clearance request; therefore there are no program changes or adjustments.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including the beginning and ending dates of the collection of information, completion of the report, publication dates, and other factors.

Upon approval of OMB, the data collection process will begin. Assuming approval in July 2009, interviews with the managers of three national laboratories will tentatively scheduled for August, September and October2009 (allowing one month for each three national laboratories). The projects will be selected during these visits. The websites for the research environment survey will be posted at the beginning of month after the visit and data collection in the three national laboratories would be completed by December, 2009. Data analysis will then be completed over the next three months, that is, by March, 2010. The contextualized reports for the first three national laboratories would be written in the months of April and May 2010. For each national laboratory a report would be presented in person to the managers of the national laboratories during the month of June 2010. The same time line would be used in FY 10-11 for the remaining laboratories.

NSF has provided funding for a third year in which combined reports will be prepared for several different audiences, including the DOE, and publications will be written.

Three sets of reports will be made:

1. Preliminary reports will be given to the three national laboratories in the first wave of data collection in 2009 and then to the second three national agencies involved in the second wave of data collection in 2010. The DOE will also receive these reports.

2. A final report on best practices for the promotion of innovation for the top managers, middle managers and project leaders will be given in 2011. The DOE will also receive these reports.

3. A final report on the implications of the research for the construction of a science of science and innovation policy will be given to NSF, OSTP, and the heads of the national laboratories. The DOE will also receive these reports.

Four outlets for publications will be targeted:

1. Science and other similar journals interested in science and research policy,

2. Evaluation journals and especially those involved in the evaluation of S&T

3. Business management journals

4. Organizational sociology journals.

Given lead-lag times involved in the writing, submitting and revising of journal articles, it is anticipated that publications will start appearing the FY 11-12.

17. If seeking approval not to display the expiration date for the OMB approval of the information collection, explain the reasons that display would be inappropriate.

All written and electronic material will display the expiration date for the OMB approval of the information collected.

18. Explain each exception to the certifications statement identified in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-I.

No exceptions requested.

12