support_stmt_a_Reinstatement no change 4-13-15

10

Supporting Statement A for Extension of

STAR METRICS®

Science and Technology for America’s Reinvestment: Measuring the EffecTs of Research on Innovation, Competitiveness and Science

Date: 13, April, 2015

Name: William Duval

Address: Office of Extramural Research, National Institutes of Health

6705 Rockledge Dr, Rm 5166

Bethesda, MD 20892

Telephone: 301 435 8683

Fax: 301 435 2003

Email: [email protected]

Table of contents

A. JUSTIFICATION

A.1 Circumstances Making the Collection of Information Necessary

A.2. Purpose and Use of the Information COLLECTION

A.3 Use of Information Technology and Burden Reduction

A.4 Efforts to Identify Duplication and Use of Similar Information

A.5 Impact on Small Businesses or Other Small Entities

A.6 Consequences of Collecting the Information Less Frequently

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

A.9 Explanation of Any Payment of Gift to Respondents

A.10 Assurance of Confidentiality Provided to Respondents

A.11 Justification for Sensitive Questions

A.12 Estimates of Hour Burden Including Annualized Hourly Costs

A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record

keepers

A.14 Annualized Cost to the Federal Government

A.15 Explanation for Program Changes or Adjustments

A.16 Plans for Tabulation and Publication and Project Time Schedule

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

List of Attachments:

Attachment 1: Memorandum of Understanding

Attachment 2: First Addendum to the Memorandum of Understanding

Attachment 3: Second addendum to the Memorandum of Understanding

Attachment 4: STAR METRICS® Research Institution Participation Guide

Attachment 5: STAR METRICS® website screenshot

Attachment 6: Privacy Act Memorandum

A.1 Circumstances Making the Collection of Information Necessary

This is a reinstatement without change of the STAR METRICS® data collection, OMB number 0925-0616.

Authority: The NIH is acting pursuant to Sections 301, 402 and 405 of the Public Health Service Act (42 U.S.C. §§ 241, 281 and 284.) See Attach 1 (Memorandum of Understanding)

Background: NIH, in conjunction with NSF, USDA, EPA and OSTP, has engaged in a partnership with universities entitled STAR METRICS®¹. STAR METRICS® is a multi-year project that was created in direct response to the reporting requirements of the American Recovery and Reinvestment Act of 2009, and aims to provide American taxpayers with accurate information on the value of their investments.

Rationale for Federal Government involvement: The Office of Science and Technology Policy, through the NSTC Committee on Science, established the Science of Science Policy (SoSP) Interagency Working Group to develop an evidence-based framework for informing policy investments in research and development, and assess the impacts of those investments broadly. Themes and goals for this emergent research field were outlined in a Federal Research Roadmap^2, and this effort was also highlighted in the FY 2011 Budget Priorities Memorandum from OSTP and the Office of Management and Budget (OMB). The memorandum asks Federal agencies to “develop outcome-oriented goals for their science and technology activities, establish procedures and timelines for evaluating the performance of these activities, and target investments toward high-performing programs. Agencies should develop ‘science of science policy’ tools that can improve management of their research and development portfolios and better assess the impact of their science and technology investments. Sound science should inform policy decisions, and agencies should invest in relevant science and technology as appropriate.”³

The initial goal of STAR METRICS® is to provide mechanisms that furnish participating universities and federal agencies with a reliable and consistent means to account for the number of scientists and staff that are on research institution payrolls, supported by federal funds.

Section 1512 of ARRA⁴ requires recipients to submit reports that contain the following

RECIPIENT REPORTS.—Not later than 10 days after the end of each calendar quarter, each recipient that received recovery funds from a Federal agency shall submit a report to that agency that contains—

(1) the total amount of recovery funds received from that agency;

(2) the amount of recovery funds received that were expended or obligated to projects or activities; and

(3) a detailed list of all projects or activities for which recovery funds were expended or obligated, including—

(A) the name of the project or activity;

(B) a description of the project or activity;

(C) an evaluation of the completion status of the project or activity;

(D) an estimate of the number of jobs created and the number of jobs retained by the project or activity; and

(E) for infrastructure investments made by State and local governments, the purpose, total cost, and rationale of the agency for funding the infrastructure investment with funds made available under this Act, and name of the person to contact at the agency if there are concerns with the infrastructure investment.

(4) Detailed information on any subcontracts or subgrants awarded by the recipient to include the data elements required to comply with the Federal Funding Accountability and Transparency Act of 2006 (Public Law 109–282), allowing aggregate reporting on awards below $25,000 or to individuals, as prescribed by the Director of the Office of Management and Budget.

STAR METRICS® was established in response to the guidance which also states federal agencies, in coordination with the Director of the Office of Management and Budget, shall provide for user-friendly means for recipients of covered funds to meet the requirements of this section. (Section G)

Past data collections: The STAR METRICS® team worked with seven universities participating in the Federal Demonstration Partnership (FDP)⁵ to identify all individuals supported by federal science funding in order to generate information on jobs created under ARRA. The consensus by the seven universities is that it is not difficult to create the required files, and there is substantial value added from standardized reports (more burden information is available in subsequent sections). The results from the pilot were presented at the FDP national meetings Sept 21-22, 2009 and January 25-26, 2010.

The STAR METRICS® proceeded with expansion beyond the original seven institutions and is currently collecting data from 100 institutional campuses.

A.2 Purpose and Use of the Information Collection

The aim of STAR METRICS® is twofold. The initial goal of STAR METRICS® is to provide mechanisms that will allow participating universities and federal agencies with a reliable and consistent means to account for the number of scientists and staff that are on research institution payrolls, supported by federal funds. In subsequent generations of the program, it is hoped that STAR METRICS® will allow for measurement of science impact on economic outcomes (such as job creation), on knowledge generation (such as citations and patents) as well as on social and health outcomes.

Since the initial approval of the data collection for STAR METRICS® in 2010, the program has reached a total number of 100 participating research campuses, testifying to its tremendous success and the perception of its usefulness by the academic institutions. There have also been two additional Federal agencies, the United States Department of Agriculture (USDA) and the Environmental Protection Agency (EPA) that have joined OSTP, NIH and NSF in the STAR METRICS® consortium.

The data elements required from the academic institutions to participate are as follows:

	Data Element	Definition
Information on Scientists and Awards	For each Award
	Federal Award Number	The identifying number assigned by the awarding Federal Agency, such as the federal grant number, federal contract number or the federal loan number.
	University DUNS Number	The organization’s 9-digit Data DUNS number
	Overhead Charged	The overhead amount charged in the reporting period.
	For each individual
	Anonymized Employee Id	Unique Employee ID (not Social Security number) of grant funded personnel
	Occupational Classification	Occupational classification / Job description of the funded personnel (ex. Faculty, Undergrad Student, Grad Student, Admin, Technical Support, Post grad Student)
	FTE Status	Designation of the status of the funded personnel (full time = 1.0, half time = .5)
	Proportion of time allocated to award	Calculated portion of the time expended by the funded personnel during the reporting time period.
Information on Overhead	Report to cognizant agency	Report submitted to cognizant agency that supports the recipients recovery of administrative overhead costs through an indirect cost rate.
Payments to vendors	For each payment to a vendor
	Federal Award Number	The identifying number assigned by the awarding Federal Agency, such as the federal grant number, federal contract number or the federal loan number.
	Vendor DUNS Number	Vendor DUNS Number. The Vendor's 9 digit DUNS number
	Payment Amount	The amount invoiced to the vendor in the reporting period
Subcontracts and subawards	For each Subaward
	Federal Award Number	The identifying number assigned by the awarding Federal Agency, such as the federal grant number, federal contract number or the federal loan number.
	Subaward recipient DUNS Number	The sub recipient organization’s 9- digit DUNS number
	Total Subaward Funds Disbursed	The amount of cash disbursed to the sub-awardee in the reporting period.

The use of the data elements: Direct jobs created and retained from science awards are calculated from four sources:

1. Individuals working directly on the project;

2. Employment within overhead

3. Employment of vendors

4. Employment of individuals on sub-awards

The calculations are provided in Attachment 4.

Use of the data The administration has committed to producing an open, transparent and auditable set of reports of the impact of ARRA on job creation. In addition, as noted in Figure 1, the media has asked important questions about the impact of federal investments in science, particularly with respect to job creation and economic growth. It is important to collect and analyze data so that such questions can be answered in a credible fashion. There is currently no data infrastructure that systematically couples science funding with outcomes.

Figure 1: Articles in Science and Nature on science job creation

Use by Universities

Universities will gather the data from their systems and transmit it to the STAR METRICS® data repository.

The first step in the STAR METRICS® data gathering process is to set up a meeting with the designated members of the university implementation team. The purpose of the meeting is to provide an understanding of the project and the information that will need to be collected with the. This first meeting may be done via webinar, teleconference, or on site and will cover the following topics:

• The STAR METRICS® team will review details of the project, discuss the information that will be collected, the process for collection, and the reports that will be generated from the data.

• The STAR METRICS® Data Dictionary will be made available to the university participants prior to the meeting so that questions may be identified ahead of time.

• Following the meeting, the STAR METRICS® team will assist the university implementation team in developing a map of the requested fields to the target sources at the university.

The second step of the process involves running the data extract routines.

• Using the data sheet as input, the STAR METRICS® team will review the mapping of the field level names against the sources for required data. The rules associated with the mapping and intended calculations from the data will also be reviewed to insure accuracy.

• The STAR METRICS® sample code library (query) examples will be evaluated for use or modification.

• The STAR METRICS® team helps select pre-defined queries or assists in building custom queries as required.

• The University team will run the queries to extract the required data.

• The extracted data will be validated with the University team

The third step in the process is to prepare the extracted data from the University systems for transmission to the STAR METRCS central repository.

• The STAR METRICS® team will help build custom translate tables as required (examples include Occupational Classification translations)

• The STAR METRICS® team will run a data cleansing routine to match federal award numbers and address any data anomalies with the University team.

• Final formatted export files will be created and validated by the University team

The fourth step in the process is to transmit the University data to the secure STAR METRICS® data repository.

• The University team will upload the files to the secure STAR METRICS® website.

• The transmitted data will be validated post successful transmission. Checks will be provided to confirm the number rows of information transmitted, the number fields transmitted per row, the format and order of the fields.

After the submission of the University data to the STAR METRICS® central repository, the STAR METRICS® team will receive the transmitted data and perform a number of data cleansing, matching and processing steps in preparation for producing summarized university reports.

The final step is to process the information that has been loaded into the STAR METRICS® database, produce and review the reports and publish to the University Web site.

• The STAR METRICS® team will run the processes that will translate and compute new variables that include: Direct jobs calculated from Individuals on a project, from overheard, and from vendors and sub-awards recipients

• The STAR METRICS® team will run jobs to create and format reports

• The STAR METRICS® team will work with the University team to validate the created reports

• The STAR METRICS® team will upload the reports to the STAR METRICS® secure website for viewing by approved University personnel

Use by NIH, OSTP, NSF, USDA, and EPA

STAR METRICS® creates a reliable and consistent inter-agency mechanism to account for the number of scientists, students and support staff that are on research organization payrolls and supported by federal funds. It will develop standardized measures of the impact of science investments on job creation stemming from grants, contracts, intramural research efforts, and other job

A.3 Use of Information Technology and Burden Reduction

The current approach to estimating job creation is extremely labor intensive and burdensome. Using a web-based approach allows the efficient and effective data collection.

Figure 2 provides a stylized description of how administrative data can be used to identify, on a monthly basis, both the universe of individuals funded by federal science agencies (PIs, coPIs, graduate and undergraduate students, lab technicians, science administrators, etc.) and the amount of federal expenditure on the individuals.

Figure 2: Using the university financial system to generate microdata

In order for the data elements to be generated, the STAR METRICS® team will need to make minor modifications of the major software packages used by the different institutions.

A Privacy Impact Assessment has been undertaken. NIH legal counsel has determined that no PII are being collected.

A.4 Efforts to Identify Duplication and Use of Similar Information

The STAR METRICS® team has presented the proposed approach to all relevant national organizations (AAU, APLU, AAMC, FDP, NAS, NCURA, AUTM) and all science agencies represented on the Science of Science Policy interagency working group. No similar system exists.

A.5 Impact on Small Businesses or Other Small Entities

The participant institutions are all universities, and are not small businesses.

A.6 Consequences of Collecting the Information Less Frequently

Initially, ARRA required quarterly reporting. We have continued that frequency since the winding down of ARRA for continuity the collection of the data must remain quarterly.

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

No special circumstances are anticipated. All the Guidelines of 5 CFR 1320.5 are met and the project fully complies

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

The 60 day notice was published on [February 5, 2015, VOLUME 80, PAGE 6522]. No comments have been received. The Science of Science Policy Interagency Group has been consulted in general. Detailed consultations have been held with the NSF, the USDA, the EPA, and with the Office of Science and Technology Policy.

A.9 Explanation of Any Payment of Gift to Respondents

Participants responding to this study will not receive remuneration for their participation.

A.10 Assurance of Confidentiality Provided to Respondents

The NIH Privacy Information Act Officer has determined that the STAR METRICS® system will not store PII and that it is not designed to retrieve information by a unique identifier. See attachment 6 for the memo from the NIH Privacy Officer.

A.11 Justification for Sensitive Questions

No sensitive questions are posed to the respondents.

A.12 Estimates of Estimated Annualized Hour and Cost Burden s

At its full operating capacity, Level I of the STAR METRICS® program includes approximately 100 university campuses. Results are reflected in the tables below.

The estimates of burden for the STAR METRICS® system are illustrated in Tables A12.1 and A12.2.

A.12 - 1 Estimate of Annualized Burden Hours
Form	Number of Respondents	Frequency of Response	Average Time per Response (in hours)	Annual Hour Burden
Ongoing quarterly data input	100	4	2.5	1,000
TOTAL				1,000

A.12 - 2 Estimate of s Annualized Cost Burden
Form	Number of Respondents	Frequency of Response	Average Time per Response (in hours)	Hourly Wage Rate	Respondent Cost
Ongoing quarterly data input	100	4	2.5	$506	$50,000
TOTAL					$50,000

A.13 Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no capital costs, operating costs or maintenance costs to report.

A.14 Annualized Cost to the Federal Government

The annualized costs to the Federal Government are expected to be approximately $806,882 per year:

Staff	Grade/Step	Salary	% of Effort	Fringe (if applicable)	Total Cost to Gov’t
Federal Oversight	14/1	107,325	25%	28%	56,882
Contractor Cost		750,000		100%	750,000
Total					806,882

A.15 Explanation for Program Changes or Adjustments

This is Reinstatement without Change of OMB number 0925-0616. The STAR METRICS® program is currently at operational capacity with approximately 100 institutional campuses.

A.16 Plans for Tabulation and Publication and Project Time Schedule

The reports will be produced on a quarterly basis, to coincide with recovery.gov reporting.

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

Forms, surveys and databases will display the OMB expiration date.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

No exemptions to the certification of the PRA submissions are requested

1 Science and Technology in America’s Reinvestment – Measuring the Effects of Research on Innovation, Competitiveness and Science

2 The Science of Science Policy: a Federal Research Roadmap, http://www.ostp.gov/galleries/NSTC%20Reports/39924_PDF%20Proof.pdf

3 M-09-27 Memorandum for the Heads of Executive Departments and Agencies, Science and Technology Priorities for the FY 2011 Budget, August 4, 2009.

4 http://206.151.87.67/docs/Section1512ARRA.pdf

5 The Federal Demonstration Partnership (FDP) is an association of federal agencies, academic research institutions with administrative, faculty and technical representation, and research policy organizations that work to streamline the administration of federally sponsored research. FDP members of all sectors cooperate in identifying, testing, and implementing new, more effective ways of managing the more than $15 Billion in federal research grants. The goal of improving the productivity of research without compromising its stewardship has benefits for the entire nation.

6 This hourly wage is estimated using the ‘Financial Analysts’ (13-2051) classification from the Bureau of Labor Statistics website (http://www.bls.gov/home.htm)