2020 BRDPI Supporting Statement Part B

Agencies are instructed to complete Supporting Statement Part B if they are using statistical methods, such as sampling, imputation, or other statistical estimation techniques; most research collections or program evaluations should also complete Part B. If an agency is planning to conduct a sample survey as part of its information collection, Part B of the ICR supporting statement must be completed, and an agency should also complete relevant portions of Part B when conducting a census survey (collections that are sent to the entire universe or population under study). For example, an agency doing a census of a small, well- defined population may not need to describe sampling procedures requested in Part B, but it should address what pretesting has taken place, what its data collection procedures are, how it will maximize response rates, and how it will deal with missing unit and item data.

Agencies conducting qualitative research studies or program evaluations, including case studies or focus groups, should also complete the relevant sections of Part B to provide a more complete description of the use of the information and the methods for collecting the information.

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Survey respondents are selected based on award levels, which determine the contributing weight of the respondent in the biomedical research and development price index.

BEA proposes to survey 150 organizations that receive NIH biomedical research awards. This sample will include the top 100 organizations in awards received and a weighted random sample (based on awards) of 50 organizations from the remaining award recipients.

Recently, survey response rates have been steady as respondents have become more familiar with the survey and BEA has made efforts to communicate the importance of the survey. BEA expects this trend to continue in the future.

Prior analysis conducted by NIH has demonstrated that there are no biases in selecting the largest award recipients—that the survey results for smaller recipients, on average is not significantly different from those of larger recipients. Thus, sampling the recipients who receive the most funding simply raises the representativeness of the survey without introducing any systematic biases.

It is also important to note that recipients receiving small award amounts from NIH are not necessarily smaller organizations in terms of the amount or breadth of their R&D activities. It is more often the case that those organizations that receive smaller funding from NIH receive, instead, larger funding for biomedical research from other science agencies, e.g., from the National Science Foundation, the Department of Energy, etc. They may also receive funds from private organizations, especially large pharmaceutical firms. Thus, the connection between size of NIH awards and size of R&D operations is extremely weak, implying those organizations that received fewer NIH awards are not likely to be fundamentally different from those that received higher levels of funding.

NIH has stated that current response rates are high enough to generate data of enough accuracy for their intended purposes. Their statement is attached below.

NIH Statement:

The National Institutes of Health (NIH) is satisfied with the survey of institutions receiving NIH awards, conducted by the Bureau of Economic Analysis at Department of Commerce. We hope that OMB will clear the survey under the Paperwork Reduction Act for another three years.

The survey provides timely, essential data on the cost structure of award recipients, which is used to estimate expenditure weights for the Biomedical Research and Development Price Index (BRDPI). The response rates are currently sufficient, and coverage is broad enough to provide data of satisfactory quality for NIH purposes. NIH would be adversely affected if the expenditure survey were to be interrupted.

The BRDPI measures changes in the weighted average of the prices of all the inputs (e.g., personnel services, various supplies, and equipment) purchased with the NIH budget to support research. The weights (including those derived from the survey of extramural institutions) are used to construct the index to reflect the actual pattern (or the proportion) of total NIH expenditures on each of the types of input purchased.

The BRDPI supports a comprehensive analysis of trends in NIH expenditures and the development of future budgets. These analyses inform policy decisions affecting the budgets for intramural labs as well as the average size of grant awards that support extramural research.

2. Describe the procedures for the collection of information including:

Once sampling is complete (see description in B1), BEA sends participants an e-mail informing them of their selection to complete the BRDPI survey. The e-mail directs respondents to report their expenditure data through an electronic instrument available at https://www.bea.gov/brdpi/; the letter and survey link are usually sent in August, with an initial due date in late October. If a completed survey is not received by the end of October, all non-respondents receive a reminder e-mail and then are personally contacted by a BEA analyst, either by phone or another e-mail. Once respondents have reported their data, BEA analysts review the data and may contact the respondent with follow-up questions, if necessary.

All price indexes, like the BRDPI, benefit by updating the weights used, which would be achieved in this case by conducting the BRDPI survey. If the collection were not conducted or conducted less frequently, then the weights applied would be those of the most recent year for which the collection was made. The longer the gap in time between the current year and the survey-data collection year, the more likely it is that the true weights could have changed significantly during that interval. If the true weights change significantly, and the old weights continue to be used, then the BRDPI would be less accurate.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

With the assistance of NIH, non-respondents are contacted through follow-up calls and e-mail to encourage response. BEA continues to follow-up with participants until a response is received or until the survey closes on December 8^th of each year. Response rates have not differed substantially across strata and have been trending upward.

To ensure accuracy and completeness, all reports are carefully examined by BEA analysts for errors and omissions.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

In 2004, nine organizations were contacted to obtain their feedback on the survey form. Seven of these organizations responded and expressed their willingness to participate in the survey. Although formal testing has not been conducted since the survey’s inception, BEA is extremely proactive with its communication with participants and answers any and all questions related to the survey. When multiple questions on particular survey items that are confusing or difficult to report are received, BEA has changed information found in the glossary (attached to the survey) in order to enhance respondents’ understanding.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The proposed survey is designed and will be conducted by the Chief of the Training and Staff Development Section, Macro Analysis and Communications Branch, Expenditure and Income Division, National Economic Accounts Directorate, Bureau of Economic Analysis (BEA).

For further information, contact:

Jennifer A. Bennett

Chief, Training and Staff Development Section

Macro Analysis and Communications Branch

Expenditure and Income Division, NEA

BEA, Department of Commerce

Phone: (301) 278-9769

E-mail: [email protected]

Attachment III: Statement B-2 Examples of Communications with Survey Participants – Initial E-mail to Survey Participants

Dear <survey participant>:

The National Institutes of Health (NIH) has engaged the U.S. Department of Commerce, Bureau of Economic Analysis (BEA) to conduct a brief survey of recipients of NIH research and development awards. The data requested will be used to update the expenditure weights needed to compute the Biomedical Research and Development Price Index (BRDPI).

As you may know, the BRDPI is an index of prices paid for the labor, supplies, equipment, and other inputs required to perform the biomedical research that NIH supports in its intramural laboratories and through its awards to extramural organizations. The index is needed for planning future NIH programs, and for developing and justifying the annual NIH budget request to Congress. For example, annual changes in the BRDPI approximate how much the total NIH budget should be increased to compensate for price increases and to sustain the level of research effort supported during the previous year. (To see last year’s report on the index, please click on this link.)

The online form, which resides on a secure BEA website, represents efforts to limit the burden of the survey. It requests only information that is absolutely necessary to accomplish the goal of producing accurate and reliable estimates of the BRDPI. Once calculated, BRDPI and some of the supporting details and analyses will be published. 

Please provide the information requested on expenditures made by your institution and reimbursed by NIH during your 2018 fiscal year. Please submit your responses to the survey provided below by

October 25, 2019.  If you have questions about this survey, please contact Evan Wang at 301-278-9674 (email: [email protected]).

Your participation is very important for maintaining the accuracy and credibility of the BRDPI, and your assistance would be greatly appreciated. Thank you for your cooperation.

Sincerely,

Michael S. Lauer, M.D.
NIH Deputy Director for Extramural Research
Office of the Director, Office of Extramural Research

To take the survey, go to:

https://www.bea.gov/brdpi

Statement B-2 Examples of Communications with Survey Participants – Follow-up E-mail to Survey Participants

This is a reminder that the deadline to submit information for the NIH survey is coming up. The survey covers funding your institution receives from NIH and is critical in computing and improving the accuracy of the Biomedical Research and Development Price Index (BRDPI.) The annual change in BRDPI indicates how much the NIH budget must change to maintain purchasing power. Participation in the survey is vital in ensuring the accuracy of the price index as well as the subsequent analyses that are based on it.

The survey can be accessed on the secure BEA website https://www.bea.gov/brdpi. The last day to submit the survey is December 8, 20xx. Due to unforeseen technology issues that might arise, we recommend and appreciate submissions from our respondents any time prior to the deadline. If you have any questions regarding the survey, please feel free to contact me.

Thank you for your time and cooperation in this important study.

Statement B-2 Examples of Communications with Survey Participants – Script for follow-up phone calls to Survey Participants

Early Fall Phone Reminder

I’m calling about the Biomedical Research and Development Price Index we are completing on behalf of NIH. With the due date coming up on _________Date_________, I wanted to make sure you have everything you need from us in order to complete the survey and if you have any questions, please feel free to contact me at ________Phone Number_______.

Late Fall Phone Reminder after e-mail

I’m following up on my previous e-mail about the Biomedical Research and Development Price Index. Our survey submission area currently shows you were unable to log on, so I wanted to make sure your institution did not have any issues logging onto the website or accessing the survey. If you need an account reset or are having technology issues, please let me know at ________Phone Number_______.

Final Phone Reminder

I’m calling about NIH’s Biomedical Research and Development Price Index. The due date is December 8^th and if there’s anything I can do to help you complete and submit the survey, please feel free to contact me at ________Phone Number_______.

Page | 7

DOC PRA ICR TOOLS MAY 2020