Change to FY 2010 Higher Education R&D Survey

Changes to FY 2010 Higher Education R&D Survey

Methodology changes

1. Expanded survey population. The survey population has been expanded to include institutions performing non-science & engineering (non-S&E) R&D. Previously the threshold for inclusion in the survey was $150,000 in S&E R&D expenditures.

Rationale: This change was made in order to capture the entire academic R&D enterprise, due to the increasing interdisciplinary nature of R&D and the frequent combination of S&E and non-S&E fields within the same R&D projects. Both user workshops and our expert panel strongly recommended collection of all academic R&D expenditures.

2. Separate reporting by campus. Each institution campus headed by a campus level president or chancellor will be asked to complete a separate survey rather than combine their response with other campuses in their university system.

Rationale: This change was made in order to establish greater consistency in the unit of measurement for the detailed statistical table rankings. Currently some state university systems such as Louisiana State University and the University of Tennessee are allowed to report as a combined total for the ranking tables, whereas campuses of other systems such as the University of Maryland are reported and ranked separately.

Overall questionnaire changes

1. All fields included in the total R&D reported in all questions. Prior to the FY 2010 survey, the main survey items only requested R&D expenditures in science and engineering (S&E) fields. R&D in non-S&E fields was requested in a separate item and was not added to the total R&D reported for each institution. Beginning with the FY 2010 survey, the total R&D reported in each question will include R&D in both S&E and non-S&E fields.

Rationale: During the data users’ workshop held in 2007, many expressed interest in expanding the scope of data collected to reflect the entire academic R&D enterprise, due to the increasing interdisciplinary nature of R&D and the frequent combination of S&E and non-S&E fields within the same R&D projects. The expert panel also recommended that the survey be expanded to include R&D within non-S&E fields in the totals reported. Over the past several years many institutional respondents have also expressed interest in integrating the S&E and non-S&E totals in the survey, to eliminate the burden of subtracting the R&D in non-S&E fields from their organized research totals.

2. Clinical trials and research training grants included in definition of R&D. Prior to the FY 2010 survey, the instructions stated that clinical trial and training grant expenditures should not be considered R&D. Beginning in FY 2010 the instructions have been revised to include phases I, II, or III clinical trials and research training grants in the definition of R&D.

Rationale: During a medical issues workshop held by NSF in 2006, the consensus of the panel was that clinical trials should be included as R&D on the survey. They were historically excluded due to interpretation of the guidance in OMB Circular A-21 for classifying organized research vs. other sponsored activities. However, since the current version of A-21 does not directly specify how to classify clinical trials, NSF decided to follow the guidance of the Frascati Manual and include phase I, II, and III clinical trials in the definition of R&D. Anecdotally, some schools were already including clinical trials in the survey because they were coded as organized research. This change is expected to aid in increasing the consistency of reporting across institutions. The exact amount of FY 2010 clinical trial expenditures (federal and nonfederal, human and veterinary) will be requested in Question 5 in order to determine the impact this addition has on the survey totals. Question 5.1 will request information on previous year reporting in order to determine the net effect of this addition to the survey totals by institution.

Although research training grants were never intended to be specifically excluded from the survey, the instruction to exclude training grants was interpreted by some institutions to include all types of training grants. Therefore the instructions have been clarified on page 3 of the survey to specifically include research training grants such as NIH K awards and T32 grants, and to exclude non-research training grants. This matches the guidance provided in OMB Circular A-21 for classifying organized research.

Individual question changes and new questions

1. Question 1, Sources of funds. Separate categories have been created for nonprofit organizations and for institutional cost sharing. The “Industry” category has been renamed “Business.”

Rationale: Previously the nonprofit source of funding was included in the “All other sources” category. Data users have long expressed interest in having this source as a separate category. Based on our site visits and the pilot test debriefings, most already have this as a separate code in their accounting systems.

The breakout of institutional cost sharing was added to the institution funds category in order to clarify what should be included in the different subcategories. Previously the instructions were not clear regarding where cost sharing should be placed, some institutions were including it under institutionally financed organized research and others were including it with unrecovered indirect costs. This change is expected to add greater consistency in the reporting of the subcategories under institution funds.

The category formerly known as “Industry” has been renamed “Business” to be consistent with the name change of the SRS Survey of Industrial R&D to the SRS Business R&D and Innovation Survey.

2. Question 2, Foreign funding for R&D. This new question requests the portion of total R&D expenditures funded by foreign sources.

Rationale: Many data users expressed interest in knowing the amount of academic R&D funded from foreign sources. Most institutions have a foreign code in their accounting systems so this was not viewed by the site visit participants or the pilot institutions to be a burdensome addition.

3. Question 3, Contracts and grants. This new question requests total R&D expenditures to be divided into two categories: contracts vs. grants and all other agreements.

Rationale: The Bureau of Economic Analysis, a major user of the academic R&D data for developing an R&D satellite account to the System of National Accounts, requested this breakdown. Based on the results of the site visits and pilot test, a large number of institutions already had these codes on the R&D accounts.

4. Question 4, R&D at medical schools. This new question requests total R&D expenditures within an institution’s medical school.

Rationale: This breakout has been one of the most common requests from data users throughout the history of the survey. Data users desire this breakout in order to better compare institutions with medical schools to those without, since having a medical school significantly increases the total R&D within an institution. The 13 pilot test participants with medical schools did not have any difficulty separating the medical school portion of R&D expenditures from the rest of the institution.

5. Question 6, Basic research, applied research, and development. This new question is an expansion of the previous question asking for the percentage of total and federal R&D that was basic research. This question requests the portions of basic research, applied research, and development for federal and nonfederal R&D expenditures.

Rationale: This additional detail is needed in order to better estimate the national totals for these categories in SRS’s National Patterns of R&D Resources. However, because these data elements are not commonly coded by institutions at the project level, the survey allows for overall estimates to be made.

6. Question 10, Other federal agency sources. This new question requests a listing of the top ten agency sources for R&D in the other category of Question 9.

Rationale: This question was added in order to track the agencies placed in the Other category of Question 9 and use the information to potentially add new agencies to Question 9 in future years. This is necessary because we do not currently have quantitative information on which agencies are the primary funders of non-S&E R&D. It will also improve the data quality of Question 9, since we discovered as part of the pilot test that some institutions were putting certain subagencies (e.g., DARPA) in the other category rather than under their appropriate agency/department category (e.g., DOD). Having such detailed information on what institutions report as “Other federal agencies” allowed us to intervene and correct the data as part of the review process.

7. Question 11, R&D funded by the American Recovery and Reinvestment Act (ARRA). This new question requests R&D expenditures funded through ARRA awards.

Rationale: This question was added to track the contribution of ARRA funding on university R&D. This specific breakdown is not available through the mandatory federal ARRA reporting system.

Previously there was a question on ARRA awards on the FY 2009 HERD pilot test and regular survey. The pilot test debriefings uncovered major inconsistencies in how award totals were reported. Despite our instructions to include the total awarded amount, some institutions only reported the funding released to them for a given year and others included the total promised amount over the life of the award. Therefore the awards question was removed from the FY 2010 survey and replaced with this question requesting FY 2010 expenditures, which is tracked by fiscal year consistently across institutions.

8. Question 12, Nonfederal sources of R&D funding by field. This new question asks for expenditures by field of R&D for each nonfederal source of funding.

Rationale: The current survey only requests field detail for federally funded R&D and total R&D. The field breakdown for all sources of funding historically has been requested by many data users who would like to have quantitative data on the R&D focus areas of the different sectors (e.g., government, business, and nonprofit foundations). Because both source and field are already coded on the R&D accounts at the majority of institutions, this expansion does not significantly add to the survey burden.

9. Question 13, Interdisciplinary R&D. This new question requests total R&D expenditures within interdisciplinary R&D centers or projects shared by 2 or more departments.

Rationale: The measurement of interdisciplinary R&D was a top priority identified during the data users’ workshop in 2007. Although the ideal was to be able to measure R&D within several interdisciplinary fields such as nanotechnology, energy, and climate research, this level of detail was not found to be feasible based on our discussions with institutions. Therefore a beginning measure of overall interdisciplinary R&D was proposed and pilot tested, with mixed success. The FY 2010 version of the question attempts to further operationalize the definition of interdisciplinary R&D to projects taking place within interdisciplinary research centers or projects that span two or more departments.

10. Question 14, Cost elements of R&D. This new question requests total R&D expenditures broken down by direct vs. indirect costs and includes categories such as salaries, software, and equipment.

Rationale: The collection of information on the cost elements of R&D was requested during the data users’ workshop as well as by the Bureau of Economic Analysis (BEA). Data users were interested in the proportion of direct vs. indirect R&D expenditures as well as the percentage going to salaries and benefits of R&D personnel. BEA is primarily interested in separately identifying software for the R&D satellite account. The data for this question was generally available for all of the pilot test institutions from the existing cost codes on the R&D accounts.

11. Question 15, Capitalization thresholds. This new question requests the institution’s software and equipment capitalization thresholds.

Rationale: This question was added in order to obtain quantitative information on the degree of variation in capitalization thresholds across institutions, since the survey requests the total spent for both capitalized software and equipment. This information is readily available within all institutions’ accounting offices.

12. Question 17, Headcount of R&D personnel. This new question requests a headcount of all personnel paid from R&D accounts, split by principal investigators vs. all other personnel.

Rationale: A top priority mentioned during the data users’ workshop was the collection of information on R&D personnel in order to help normalize the R&D expenditures by the size of the R&D “workforce” at the institution. The collection of headcounts as well as FTEs (full time equivalents) for R&D personnel is also a data element recommended by the Frascati Manual for international comparisons of R&D.

A more detailed question on R&D personnel (FTEs by field of R&D) was cognitively tested but deemed too burdensome by the majority of respondents interviewed. Therefore a less burdensome question requesting headcounts associated with the R&D salaries reported in Question 14 was pilot tested with much greater success. The majority of institutions was able to provide the headcount and had codes identifying which personnel were PIs.

13. Question 18, Headcount of R&D postdocs. This new question requests the number of postdocs within the total headcount reported in Question 17.

Rationale: This question was added to complement the work being performed to count postdocs via the Postdoc Data Project within SRS. The counts provided to the HERD survey will represent a subset of the total postdocs at an institution, since it only includes those paid from R&D accounts.