Part B OMB supp statement for HERD

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. The Universe and Response Rates

The Universe

The HERD and FFRDC R&D Surveys are censuses of all eligible institutions. No sampling will be performed for either survey.

The universe for the FY 2019-21 HERD Survey population will include all bachelor’s and higher degree-granting U.S. institutions that spent at least $150,000 in separately accounted for R&D in their prior fiscal year, estimated to be roughly 950 institutions each year. The population for the survey will be identified through a screening of all bachelor’s and higher degree-granting institutions as identified by the National Center for Education Statistics Integrated Postsecondary Education Data System (IPEDS). All institutions reporting at least $150,000 in total R&D expenditures will be included in the survey population.

The universe for the FFRDC R&D Survey consists of all FFRDCs in operation during the fiscal year of measurement (42 in FY 2018).

Response Rate Calculations

The AAPOR response rate 6 (RR6) is used to calculate the overall survey response rate (https://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf page 62). This response rate includes complete plus partial responses in the numerator divided by all known eligible institutions. The response rate is expected to be over 95% each year for the HERD survey, and 100% for the FFRDC R&D Survey. Previous response rates for the HERD survey are 97.1% in FY 2016, 96.2% in FY 2017, and 96.9% in FY 2018. The FFRDC R&D Survey has consistently obtained 100% response each year.

2. Description of Survey Methodology

The HERD and FFRDC Surveys will be administered via the web. Prior to the survey launch, a population review will be conducted to identify newly eligible institutions for the surveys. For the FFRDC Survey this involves a review of the current NCSES Master List of FFRDCs to update the population as needed. For the HERD survey, this involves contacting all four-year and above institutions reporting non-zero R&D expenses in IPEDS, who were not previously included in the HERD survey population and had not already responded to a screener in the previous year. The number of academic institutions contacted is estimated to be approximately 150 per year. These institutions will receive an email with a pdf attachment of the Population Review questionnaire. This brief survey will ask whether the institution spent over $150,000 on R&D projects during either of the past two fiscal years, and if so, whether the expenditures were estimated to be between $150,000 to $1 million or $1 million or more. Institutions responding with expenditures over the threshold will be included in the HERD Survey or HERD Short Form depending on the expenditure range reported. Institutions who do not respond after two bi-weekly reminders will be contacted by phone as needed.

An initial contact verification email will be sent to all previous year respondents. One week later the respondents will be sent survey information and a user ID and a link with instructions to create a password via email. Each institution’s web survey includes data from the 2 preceding years for arithmetical and trend check purposes. Large increases or decreases in expenditures are compared with the preceding years and flagged, and respondents are asked to explain and reconcile these differences.

Telephone and email follow-up will begin approximately 2 weeks after survey transmittal to those respondents who have not verified they are the correct contact for their institution. Reminder mailings, emails and telephone calls will be made at several points throughout the survey collection period. All telephone contacts include an offer to assist the respondent in providing the data. NCSES and the contractor will monitor institution response status closely through a web-monitoring system during the data collection cycle, with periodic contact maintained via telephone and email until survey closeout to ensure that (a) a response is provided and (b) data errors are resolved. This level of interaction with the institutions is responsible for the high survey response rates that have been achieved for the past years (see Attachment 11 for the full set of contact procedures).

Upon survey closeout, NCSES will develop estimates for that portion of the survey population that did not respond. Imputation is performed using prior years' figures derived from the data of respondent institutions with similar characteristics, such as highest degree granted and type of institutional control (public or private). This process has been used consistently since 1976. As a result of high overall response rates, especially from institutions that historically account for the largest share of the HERD total, imputed amounts account for a very small percentage of total R&D expenditures (0.1 percent of the total in FY 2017).

3. Methods Used to Maximize Response Rates

All of the follow-up procedures described here and in the previous section have been instituted in part to maximize the survey’s response rate. While the official survey deadlines are three months after the opening of the survey, the data collection window is held open as long as possible to obtain responses from all of the FFRDCs and at least 95% response for the HERD Survey (including all of the top 150 institutions in R&D spending in the prior FY). Institutions that do not respond by the deadline are contacted every one to two weeks either by email or phone to remind them of the importance of completing the survey for their institution. For the HERD Survey, if an institution has still not responded approximately six weeks after the survey deadline, a letter is sent to their president or chancellor making one final request for their participation. These methods have delivered a response rate of 94% or higher approximately eight weeks after the initial survey deadline for the last several survey cycles.

4. Tests of Procedures

There are no plans to conduct tests of procedures during this clearance period.

5. Names and Telephone Numbers of Individuals Consulted

The FY 2019-21 HERD and FFRDC Surveys will be conducted by NCSES, under the general direction of Mr. John E. Jankowski, Program Director, Research and Development Statistics Program (703-292‑7781) and under the daily direction of Mr. Michael Gibbons, Survey Manager for the HERD and FFRDC Surveys (703-292-4590). Reports will be prepared by NCSES staff.

ICF is the contractor in charge of data collection and tabulation. The senior project officer for the contract is Kate Flint (301-572-0333).

Professional statisticians and survey methodologists on the NCSES staff are also available for review and advice, such as Jock Black, Mathematical Statistician, and Rebecca Morrison, Survey Methodologist.

Copies of cited documents are available from Mr. Michael Gibbons, Survey Manager (703-292-4590).

ATTACHMENT 1

OMB Notice of Approval 3145-0100

ATTACHMENT 2

FY 2019 Higher Education R&D Survey questionnaire

ATTACHMENT 3

FY 2019 Higher Education R&D Short Form Survey questionnaire

ATTACHMENT 4

FY 2019 Higher Education R&D Survey population review questionnaire

ATTACHMENT 5

FY 2019 Higher Education R&D Survey Population Screening Methodology

ATTACHMENT 6

FY 2019 FFRDC R&D Survey questionnaire

ATTACHMENT 7

First Federal Register Notice for the FY 2019 HERD Survey

ATTACHMENT 8

HERD Comment from the Bureau of Economic Analysis

ATTACHMENT 9

HERD Comment from the University of Washington

ATTACHMENT 10

HERD Comment from the University of Wisconsin

ATTACHMENT 11

Draft contact materials for the FY 2019 Higher Education R&D Survey