LFM with comments included

-AIMS Draft LFM 5_4_16 Questions_Comments LFO Responses 08-17-16
-AURA comments on draft LFM2016
-Coles Comments on LFM 2016 Draft 160701
-Falkner-LargeFacilitiesManual2016Draft_5 4 2016
-Graff LFM Errata_ Commissioning Plan and _IMP_
-Graff LFM, minor addition_ Section 3.4.1-10.2
-Graff NSF LFO Improvement Suggestions 2016-06-23
-Hawkins-Wilkinson Re_ Contingency for MREFC Fixed Price Contracts
-McGovern Comments on Large Facilities Manual May 2016
OIA _'No comment' on draft Large Facilities Manual
-OIG Audit Comments on NSF's Large Facalities Financial Data Collection Tool
-OIG Comments on Draft Large Facility Manual
-Porter LFM PEP Guidance & Draft Comments_
-Santoro 2.2.2
-Sholhead comment_ LFM draft
-Whitehurst_ LFM GAO-CO
-Whitehust RE_ LFM 'termination'
-Wilkinson Page from Large Facilities Manual (LFM) - Draft May 2016

Meeting Date: 8/5/2016

LM AIMS team questions on Draft LFM 5_4_2016
General Statement: A majority of the LFM appears to be geared toward cooperative
agreements and not for our situation.
1. Pg 2.1.3-1. Points at which there may be departure from the MREFC process outlined
here should be identified early in the project development and documented as part of
the NSF Internal Management Plan (IMP) Individuals should discuss any proposed
departures with the cognizant Program Officer.
a. Question: Will there be any documentation that the Operations and Divestment
stage are not part of this project? The document says any departure from these
stages would be identified in the NSF IMP but I wasn’t sure if that information
was reflected.
2. Pg 2.1.3-3. Preliminary Design Phase Definition: Further advances the project baseline
definition and the Project Execution Plan. It produces a bottom-up scope, cost, schedule,
and risk analysis of sufficient maturity to allow determination of the Project Total Cost
and overall duration for a given Fiscal Year start and to establish the MREFC budget
request to congress.
a. Question: It is understood we provide a funding profile and spend plan for PDR.
We also understand we baseline at PDR per the LFM for scope, cost and schedule.
Does the LFO allow for changes in the funding and spend plan profiles at the end
of FDR to allow for any updates we may need to make based on the awarded
subcontract?
3. Pg 2.1.6-5. Table –

a. Who is the PO – Scott or Ben? It was confirmed at our meeting on 8/5/16 that Ben is
the Program Officer in our case. What do they mean by the CO “creates

solicitations for enabling research, workshops, summer studies…. (the first
1

Meeting Date: 8/5/2016

bullet). 8/5/16 Scot confirmed the first bullet was more for cooperative agreements
and not required for us.

b. For those items highlighted yellow:
i. Does the PO have authority to approve significant change to the project
baseline? 8/5/16: Response was in our case the PO does not authority to
approve baseline changes.
ii. Who are the external ad hoc panel members? 8/5/16: Requires further
review.
iii. Need to understand the “periodic reviews” of project progress – monthly,
quarterly, annually so we can determine frequency and requirements for
these reviews? 8/5/16: This has not been determined.
iv. Is the GPRA where the requirement for an annual performance plan
comes from? 8/5/16: Requires further review and verification by NSF.
4. Pg 2.1.6-13. The G/AO is responsible for oversight of the financial and administrative
terms and conditions of the assistance agreement,1 just as the PO is responsible for
scientific and technical oversight. Unlike the PO, he/she holds the warrant to obligate
government funds. The G/AO and the PO jointly share the principal technical and
financial responsibilities for the oversight and assurance of a large facility project. In this
capacity, the G/AO is jointly responsible with the PO for the success of a project.
a. What is the difference between the G/AO and the CO?
5. Pg 2.1.6-14. The CO holds the warrant and is the only individual authorized to obligate or
de-obligate government funds. The CO, through their warrant, has the sole authority to
2

Meeting Date: 8/5/2016

award and administer the construction contract(s) used in support of Large Facility
projects.
a. This statement says the “CO has the sole authority to award and administration
the construction contract (s) used in support of Large Facility projects. This does
not make sense for our situation – can NSF clarify the CO’s role in our case.
6. Pg 2.1.6-14. Cost Analyst. The G/AO or CO requests assistance from a NSF Cost Analyst from
the Cost Analysis and Audit Resolution (CAAR) Branch of the Division of Institution and
Award Support (DIAS), located within BFA (CAAR) when cumulative or individual awards
exceed certain thresholds or for Recipients with previously identified risks. The PO, G/AO or
CO, and Cost Analyst all review proposed budgets to help determine if they are allowable,
allocable, reasonable, and realistic for the scope of work. However, the primary purpose of
the NSF Cost Analyst’s budgetary review is to support the G/AO or CO to ensure that the
Recipient has properly estimated and calculated costs and that they are supported and
documented with sufficient rigor. The Cost Analyst provides a written recommendation to
the G/AO stating whether costs are supported or unsupported. The recommendation may
include advice on award terms and conditions or limitations or other concerns identified.
The Cost Analyst may also help determine if the Recipient has adequate business and
accounting systems in place, assess a Recipient’s financial capability and viability, validate
indirect cost rates, or assist in other areas of concern as identified by the requesting G/AO.
a. Who is this on NSF side? 8/5/16. Need to confirm with Kevin that this is not a
cost item that NSF must bear and verify if this person is in place right now.
7. Pg 2.1.6-14 (Footnote).
1 Refer

to the Business Systems Review (BSR) Guide described in Section 4.5.3.3 for discussion on
this point. When NSF is not the cognizant audit agency for the recipient institution, its oversight
of recipient business practices is narrowly defined.

a. Will there be another BSR on ASC as there was already one
conducted when the current contract was awarded? We need to
understand timing as well for when this will take place.
8.

Page 2.3.3-1 4th bullet and 2.3.3-2 footnote comment: A fully implemented PMCS, including
a final version of the resource-loaded schedule and mechanisms for the project to generate
reports – using the Earned Value Management System (EVMS)1 – on a monthly basis and
use them as a management tool. Path dependencies, schedule float, and critical path are
defined; and footnote 1 During construction, progress should be tracked and measured using the Earned Value
method (this method is required by OMB in Planning, Budgeting, Acquisition, and Management of Capital Assets, OMB
Circular No. A–11 (2014). A discussion of Earned Value is included section of Section 8, Guidelines for Earned Value
Management.

a. Will this be a requirement for LM which we will then flow down to our
subcontractors? This is a requirement for the prime contractor (or the award
3

Meeting Date: 8/5/2016

recipient for cooperative agreements). Subcontractors must be able to provide
the appropriate cost and schedule inputs to put into the prime’s EVMS. So this
would apply to LM, but not necessarily subcontractors. (Please note, this
requirement is not new to the 2016 draft, it has been in the LFM in similar form
since at least 2009. Additional guidance for how projects are expected to meet
this requirement is the part currently being developed.) 8/5/16 - Need
definition between certification and validation from the power point
presentation we were shown and which is required and by whom (government
audit agency or third party). Also need to understand how the FAR 34.2 and
DFARS 234.2 words compliant and non-compliant differ from certification and
validation. NSF guidance is being developed to require validation of EVMS to
the intent of the EIA-748 standard. This guidance will explain how the EVMS is
validated for NSF MREFC projects. NSF will not require that EMVS be certified
by DCMA. An expert third party will validate the EVMS meets the intent of EIA748. The EVMS section of the LFM will be comprehensively updated during the
2017 review cycle. However, in the interim, NSF standard operating guidance
will be provided to projects. LFO will also consider adding some clarifying
words to this draft of this 2016 LFM to (1) require validation of the project’s
EVMS to the intent of the EIA-748 standard and (2) allow projects to request
minor deviations with supporting justification. What are the ongoing monthly
ASC (and subcontractor flow-downs) EVMS
requirements/deliverables/expectations? (ie. Will the government auditing
agency or 3rd party agency continue to audit the project monthly, meaning we
deliver all month end business files (including subcontractor?), respond to
findings, work corrective actions, etc. together with NSF EVMS person and
EVMS auditing agency or will the EVMS system be “validated” 1x for the project
and monthly EVMS reporting will continue as per our current
rhythm/deliverables on the ASC contract? The LFM requires LM to provide
monthly EVM reports for construction. Subcontractors must be able to provide
the appropriate cost and schedule inputs to LM for incorporation in LM’s EVMS.
An initial validation will be done in conjunction with FDR to allow resolution of
any findings prior to the start of construction. The validation will include a
“desk review” of plans and procedures followed by a visit to the project to
interview project controls personnel and review the system. Future periodic
surveillance by a third party is expected to be done annually, unless findings
require more immediate or routine follow-up until resolution. If the agency/3rd
party EV company is on contract for the duration of the project, who is the
EVMS NSF contact that ASC would deliver monthly deliverables to and ensure
compliance/work off findings? Expert third party will be contracted by NSF. No
expectation for monthly deliverables except the EVM report per the LFM. LM
should have the appropriate PMO to maintain the PMCS, including EVMS, and
4

Meeting Date: 8/5/2016

issue monthly reports. When does the EVMS system need to be “validated”? Is
it a constraint to exiting PDR/baselining, or required prior to FDR (we will
already be baselined)? In conjunction with FDR, to allow resolution of any
findings prior to the start of construction. Having the EVMS up earlier is
prudent to ensure it is working properly. If EVMS findings are required post
initial baseline, will we have an opportunity to clean up and re-baseline? If
findings are significant and meet criteria in LFM section 4.5.5, yes possibly. But
it is not obvious why a future surveillance of the EVMS might prompt a rebaselining. Note: It is the ASC recommendation that EVM
compliance/validation not be a mandatory subcontract flowdown. Rather ASC
will define requirements for subcontractor submittals and enter the data into
the ASC EVM complaint system. Agree, this has been done successfully for
other projects. This requires clear requirements in subcontracts (e.g., for
timely, discrete cost and schedule inputs that can be physically verified). A
RESPONSE TO THIS IS CRITICAL TO ENSURE WE CAN MEET PDR SCHEDULE AND
INCORPORATE ANY ADDITIONAL COSTS REQUIRED DURING CONSTRUCTION
DEPENDING ON EVMS DETERMINATION. Ben emailed Kevin to set up a
meeting for this specific topic.
b. The bullet would indicate that we are to provide a “final version of the resource
loaded schedule” during FDR. Does this mean we are allowed to update the
“project baseline that is set following PDR (pg 4.2.5-3)? We understand our Total
Project Cost would not change but are we allowed to re-baseline our entire IMS
schedule once we receive a refined schedule from our awarded subcontractor? Is
the intent that the Project (from total project cost and high level scope/schedule
– buildings in/buildings out) is “baselined/locked down” at PDR, but the actual
IMS (detailed tasks, resources, cost spreads/allocations) is not baselined? If so, is
that PDR baseline to remain intact per the “NSF No Cost Overrun” direction for
the entirety of the project? If, at FDR, we learn the General Contractor
bids/schedules are significantly different (more than 10% cost or schedule) than
what we planned and baselined at PDR, are we able to re-baseline to the GC plan,
or are we held to the plan we established at PDR? If/when re-baselining is
necessary, what is the process/timeline to work with the NSB to re-baseline
(approvals, any change boards mtgs/formal change presentations, paperwork) in
order to not compromise delay in funding or in construction on the ice? If we
have to exercise risk contingency, can NSF approve or does the NSB also need to
approve? Can the project continue working when a risk event is realized
exercising risk contingency or is the program on a stop work pending NSF/NSB
approval/exercise of risk item?
c. Are we allowed to carry an amount of “schedule contingency” (ie. time in the
form of an actual “schedule contingency” task) in the IMS? Are we allowed to
carry those schedule contingency tasks throughout the IMS (completion of each
5

Meeting Date: 8/5/2016

building) or a total amount attached to the end of the project? Which date do we
report as our estimated complete date? Or both dates?
9. Pg 3.4.2-1
a. Can we get the additional details NSF PO expects to see in the PEP?
10. Pg 4.2.1-2, NSF requires 90-180 days to do a cost analysis before we progress to the next
design phase, yet the figures 4.2.1-1 reflects this analysis is done post FDR but prior to
construction award. 1 – It sounds like NSF intends to perform this prior to FDR, is this
accurate and if yes why the deviation from the LFM. 2. Does NSF expect to do a cost
analysis of our subcontractor proposals (IAW 4.2.1) prior to award of construction? THIS IS
ANOTHER CRITICAL ITEM AND REQUIRES IMMEDIATE DISCUSSION AND RESOLUTION. (LM to
provide graphic)
PJJ Comment: It is my assumption that in order to complete a cost analysis post FDR but
prior to award of construction (CLIN 005) we will need to have our subcontractor
proposals received, down select, site visits, final proposals, orals, negotiations and LMs
analysis. After all of that, then NSF will start their analysis of the LM submittal which will
take up to 180 days. It will take LM 30 days to finalize our review and analysis, NSF 180
days to do the same. Then at least 30 more days for the NSF to issue the modification to
our prime contract and 20 days for LM to process the award once funding has been
received – this is 260 days or 8.6 months. This is an extraordinary amount of time for
construction proposal validity period. Three months is often at the high end of a validity
period range. However, extending this period will cause subcontractors to price in
additional risk due to cost of materials, resource availability, etc.
11. Pg 4.2.2-7 Section 4.2.2.4, G.5 –
a. 8/5/16: Clarification on the reference to uniform guidance, 2 CFR section #200.331 is
incorrect so needs to be fixed. GPO.gov says parts 305-399 are reserved. Would like
clarification on intent of this reference or an alternate source document for its
availability. https://www.gpo.gov/fdsys/search/pagedetails.action?collectionCode=CFR
&searchPath=Title+1%2FChapter+III%2FPart+304&granuleId=CFR-2014-title2-vol1sec200-331&packageId=CFR-2014-title2vol1&oldPath=Title+1%2FChapter+III&fromPageDetails=true&collapse=true&ycord=479.
12. Pg 4.2.2-8 Management Fees
a. 8/5/16 - This needs to be clarified for situations involving contracts. This addresses
cooperative agreements. Contracts need to be addressed differently or its specific
exclusion noted in the IMP to avoid confusion (see question 1 and Pg 2.1.3-1).
13. Pg 4.2.3-3 and Figure 4.2.3-2.
a. 8/5/16 - Are we required to provide the cross-walk to the NSF budget format/category
code?
6

Meeting Date: 8/5/2016

14. Pg 4.2.3-5 Construction Cost Book Detail and Supplementary Guidance
a. Is this how they want to see our subs proposals? Do subs need to submit BOEs for each
labor category? Do they need to use NSF budget category codes? If yes, do we want to
have the GC use A-I or just G-5 for all BOEs (this section refers back to section
4.2.2.4)? While I am not used to doing this for construction it is not impossible. It
should be noted that in addition to BOEs there will be need a detailed cost estimate by
WBS in excel to match up with each BOE.
i. 8/5/16 - Requiring a General Contractor to provide this level of detail for each labor
category seems unnecessary. This may be more appropriate for contracts that are not
fixed price.
15. Pg. 4.2.5-1 – Discussion on Budget Contingency Planning for the Construction Stage.
a. This section states, “In support of NSF’s “No-Cost Over-run” policy, confidence levels
(does “confidence level” mean high level total project cost and high level scope (buildings
in/buildings out), or detailed IMS level tasks, resources, cost spreads?) must be in the 7090% range when the project baseline is set following PDR depending on the nature of the
project; including the ability to de-scope.
i. Subcontract awards for the construction will not be finalized by PDR so requiring a 7090% confidence in our baseline at this time may be challenging. Is it up to the PO to
either de-scope or use any budget contingency if subcontractor pricing comes in
above the estimate.
16. Pg 4.5.3-2– What BSRs is NSF planning to do and when?
17. Pg 4.5.3-3 – This is a general question but is NSF considering all three buildings one award or
each building individually? Does the NSF plan to perform an incurred cost audit on any
subcontractor award that exceeds $100M?

7

ASSOCIATION OF UNIVERSITIES FOR RESEARCH IN ASTRONOMY, INC.

Suite 1475
1331 Pennsylvania Avenue NW
Washington, DC 20004
TEL: 202-483-2101
FAX: 202-483-2106
OPERATING FOR THE
NATIONAL SCIENCE FOUNDATION

July 8, 2016
National Science Foundation
4201 Wilson Blvd.
Room 1265
Arlington, VA 22230

Gemini Observatory
La Serena, Chile & Hilo, Hawal'I

Attention: Suzanne Plimpton - Reports Clearance Officer

Large Synoptic Survey Telescope
Tucson, Arizona & La Serena, Chile

Subject: Draft 2016 Large Facilities Manual

National Optical Astronomy Observatory
Tucson, Arizona & La Serena, Chile

Dear Ms. Plimpton:

National Solar Observatory
Boulder, Colorado & Maui, Hawai'i
Sunspot, New Mexico & Tucson, Arizona
OPERATING FOR THE NATIONAL AERONAUTICS
AND SPACE ADMINISTRATION
Space Telescope Science Institute
Baltimore, Maryland

In accordance with the May 9, 2016 Federal Register Comment
Request, AURA is providing the attached comments regarding the
subject Large Facilities Manual draft. We hope you will take the
attached remarks into consideration when finalizing this document.
Please do not hesitate to contact me with regards to any questions
you may have on our feedback.

MEMBERS/SINCE:
Boston University 1993
California Institute of Technology 1972
Carnegie Institution of Washington 1997
Cornell University 2016
Fisk University 2010
Georgia State University 2008
Harvard University 1957
Indiana University 1957
lnstituto de Astrofisica de Canarias 2005
Iowa State University 1992
Johns Hopkins University 1982
Kiepenheuer-lnstitut fUr Sonnenphysik 2016
Massachusetts Institute of Technology 1981
Michigan State University 1997
Montana State University 2005
New Jersey Institute of Technology 2010
New Mexico State University 1999
Ohio State University 1957
Pennsylvania State University 1990
Pontificia Universldad Cat611ca de Chile 1997
Princeton University 1959
Rutgers University 1999
Stanford University 2012
Stony Brook University 1986
Swinburne University 2008
Texas A & M University 2014
Tohoku University 2010
Unlversidad de Chile 1992
University of Arizona 1972
University of California Berkeley 2007
University of California Santa Cruz 1957
University of Chicago 1g57
University of Colorado 1977
University of Florida 2002
University of Hawaii 1978
University of Illinois 1980
University of Maryland 1986
University of Michigan 1957
University of Minnesota 1995
University of North Carolina at Chapel Hill 1995
University of Pittsburgh 2012
University of Texas at Austin 1972
University of Toledo 2016
University of Virginia 2003
University of Washington 1986
University of Wisconsin 1957
Vanderbilt University 2010
Yale University 1958

Sincerely,

Val Schnader
Vice President for Operations
Phone: 202 204 1373
Email: [email protected]

AURA comments on draft LFM 2016

Table 3.4.1-1, List of the Essential Components of a Project Execution Plan
Section 4.4 of the components table "Scope Management Plan and Scope Contingency"
includes "up-scoping opportunities" remarks. The intent of this phrase should be clarified, as it
is unclear if this relates to the recovery of prior de-scopes or allows true scope increases.
Figure 4.2.2-1 Sample Project Management Control System Flow Chart

Regarding Risk Analysis Tools, the flow chart only shows only one approach; Monte Carlo
Simulations. We recommend that other approaches be documented (e.g. bottoms up
methodology using Cost, Schedule and Technical risk factors)
Section 4.2.2.4 Supplementary Guidance for Construction and Operations Awards.

To preclude confusion and to ensure consistency with Section 2.3.3.3 language, the word "subawards" should be replaced with "sub-awards and subcontracts" in both sub-sections G.5 and
G.6 of Section 4.2.2.4. Given that the OMB Guidance addresses the terms "sub-awards" and
subcontracts" separately, and noting that Section 2.3.3.3 of the Manual also addresses the
terms separately, we suggest that Section 4.2.2.4 should read in the same manner.
Provision 5 on Section 4.2.5.1 states "NSF will hold budget contingency through project
completion, in an amount up to 100% of the total NSF-approved contingency budget, until it
can be justified for obligation."
It should be noted that the NSF has already approved the contingency budget through the
rigorous exercise noted in Section 4.2.5, so clearly the contingency amount (in aggregate) has
been justified. In addition, as per 'standard' Cooperative Support Agreements, the NSF requires
"All proposed changes or proposed use of contingency will be made known to NSF (posted via a
change control database or similar) prior to authorization." It would thus appear that
justification before approval is already established. It is not clear what additional justification
is required by this provision.
Also, it is unclear from this language as to whether the contingency funds will be provided as
part of the annual funding action consistent with the funding profiles already established as
part of the NSF process. If it is not, then there would potentially be delays in timely receipt of
funds for contingency events; this could increase project costs. This change could restrict the
ability of the projects to proactively manage risk (since funds may not be available as planned
when needed - this again would drive project costs).
With regards to determining the amount to withhold, the burden to predict usage and the
amount of contingency to allocate is a new administrative task that has the potential to drive

new tools or new features to existing tools. The new NSF expectation for risk levels and project
health to be verified through Monte Carlo analysis of integrated cost, schedule and risk plans is
not immediately suited to a time phased assessment. If historical performance and algorithmic
models can be used this burden will be minimized but this policy is a noticeable increment in
periodic administrative effort
Section 4.5.3 Recipient Performance Reviews and Audits

The review processes documented in Sections 4.5.3.2 through 4.5.3.4 are in essence "open
ended" in number of reviews that may take place. It would be beneficial for the Foundation to
establish some estimate of expected effort required by the awardee to support these activities.
For programs already in place wherein project costing did not assume such required support,
consideration should be provided for additional funding to address this additional scope.
In addition, the BSR explanation notes that the BSR " ... determines if these policies and
procedures conform to OMB requirements, NSF expectations, and other applicable federal
regulations." While the OMB requirements and federal regulations are explicit documents, it is
not clear where "NSF expectations" are defined. This lack of clarity extends to the BSR Guide,
and needs to be addressed more specifically.

The following are additional comments regarding the Large Facility Financial Data Collection
Tool:

The instructions on the form are unclear as to whether a form for each CSA under a CA is to be
submitted or if one form for fill CSAs under a CA is required.
Additionally, the guidance does not provide specific deadlines requirements. Currently, it
references to the award terms and conditions. However, the award terms and conditions do
not address this compliance requirement.
It is not yet clear when the requirement for submitting the 'first' report is, if for FY16 data that
could create issues. NSF is asking for public comment that will certainly lead to some changes in
the data collection tool and will take time for NSF to revise. Given that we are already into the
FY 4th quarter, there likely will not be sufficient lead time to properly prepare once the revised
tool is released.
Tab: Instructions

Cell 10 - Instruction do not clearly define if awardees are expected to insert summarized
expense total in the cells Users are to complete, or if detail (raw data) behind the summarized
expense totals is to be provided. The draft spreadsheet allows for new tabs to be added which
could facilitate the importing of detail, but it is unclear if the draft spreadsheet works in the
same manner the final version will.

Cell 12 - This cell implies that multiple CSAs should be reported in the spreadsheet tool and the
data will be consolidated on this worksheet. Conversations with NSF personnel at the Large
Facility Workshop indicate that a separate spreadsheet tool should be submitted for each
separate CSA. Clarification is required on this matter. Please note that AURA has separate
CSAs under the same NSO CA that are funded from NSF funds and MREFC funds and we suspect
these should not be reported as a combined total.

Subcontract & Consultants Worksheet (comments apply to Sub-awards worksheet as well)-Data as outlined on contracts is not required from the vendors in this format, so now requiring
this data would result in additional cost for both the awardee and the vendor.
-Having this kind of detail in the proposal\budget stage is not part of current process.
-Having to gather data for each contract paid during the reporting period could result in an
unmanageable amount of data.
-Not all contracts provide this information, requiring it could cause additional expense raising
prices of services or goods.
-Many contractors that we work with are for profit companies so their costs may not reflect the
price we pay for services; how do we report profit?
-Verification that the data is accurate will be difficult or costly.
-Time involved in gathering, storing and organizing this data would require additional effort by
the awardee.

Setup tab -The cell in which the Users is to fill in the number of Subcontracts or Subawards
being reported will not take a number greater than 7.
If Other, list name below:
Na m e 1:
Na me 2:
Na m e 3:

-

--

-

-

-

Na m e 4:
Na m e 5:

=
r-

-- -

Number of Subcontracts cannot be Greater than 7

Nam e 6:

CSA
CSA 1

Run Setup

-

--

#of Su lJcontracts

# ofSubawards

10

10

Number of Subawards cannot be Greater than 7

OK

OK

There will likely be situations where there are more than 7 contracts totaling a minimum of
$150,000. Either remove the limit and allow input for all contracts above 150k or explicitly
state if contracts should be chosen by highest BAC, highest BCWS, highest Actuals, etc. for the 7
being chosen.
For grand totals to tie out for the total project all subcontract values should be used. Only
selecting contracts totaling a minimum of $150,000 could leave a substantial gap in reporting
the total BAC of the project and may cause confusion validating data. Perhaps a solution is one
line for all other contracts less than $150,000.
The input sheets need to be clearer if budgets being entered are cumulative to date or BAC.
Reducing the ambiguity of the budget data being requested would be helpful (specifically call
out if this is a FY request, cumulative to date, or BAC). It may be useful to populate the tool
with a sample (test) data for examples of how the budget and actual totals trace across the
worksheets.
There are some cases where cells should be merged on the Subcontract/Sub award sheets (i.e.
D:E, F:G, etc. for general look and feel). It also seems like rows 31 and 32 on those sheets
should be formulas from the data entered above (Gray). Just a note, there could be a
disconnect with this tool and EVMS data as EVMS data also includes estimated actuals based on
delayed accounting processing of invoices.
Cooperative Support Agreement (CSA) Budget Worksheet -

Guidance needed for the level of supporting documentation detail required to provide for
"total expenditures to date" on existing CSAs. Is the first report submitted supposed to report
inception to date expenses as the current year expenses?
On the CSA Budget worksheet, it is unclear if this should only include the 7 Subawards listed
from previous sheets or if this is a grand total. This is the problem with only entering a subset
of data for the Subcontract/Subawards as there will be a delta between total budget numbers
for the CSA and the numbers displayed in other sheets.
General questions on data to be provided:

Are totals in this report intended to tie to any other report submitted to NSF?
Program reports include data on commitments which are not reported here.
ACM$ reports only show cash drawn, some expenses reported here may not have been
paid in cash yet.
Audited financial statements
How is program income reported?

Comments on LFM 2016 Draft
1. I did not find this: Changed issuance of the HLFO MREFC facility status report from “monthly” to
“periodic” on page 2.1.6-13. Whether or not HLFO issues status reports, the spending rate for
MREFC projects motivates NSF Program Officers and other staff receiving technical and financial
status reports on a monthly basis during construction.
2. Table 2.1.6-1 says the PO is “Experienced or trained in management of large projects.” It also
says the LFO Liaison “Advises POs on project management issues during project development
and oversight” yet there is no requirement that the LFO have similar experience or training.
3. Change 5 refers to a third bullet on page 2.2.2-9, yet there is no page 2.2.2-9. (The manual goes
from 2.2.2-1 to 2.3.1-1)
4. Section 4.2.2.2 “Relevant Guidance” needs editing. It says: “Proposed budgets must comply with
the applicable federal regulations, as implemented by NSF in the Proposal and Award Policies
and Procedures Guide (PAPPG), including Grant Proposal Guide (GPG) and Award and
Administration Guide (AAG). Recipients are required to follow the best practices within the GAO
Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital
Program Costs (GAO-09-3SP March 2009, or subsequent revision) and GAO Schedule
Assessment Guide: Best Practices for Project Schedules (GAO-16-89G December 2015, or
subsequent revision), taking into consideration NSF policy and practice as provided in this
manual. Recipients must note any departures from these GAO guides and explain their
rationale.” The first part doesn’t need saying – as NSF never tells proposers that they do need to
comply with federal regulations. As the title is “Guidance”, instituting the requirement to follow
best practices restricts flexibility that can be potentially beneficial in a scientific research
environment. The logic of the paragraph is further tangled when it concludes that the recipients
apparently are not required to adhere to best practices but can instead explain their rational for
deviating. (I support that flexibility – but the logic of the paragraph is a muddle and needs to be
rewritten coherently.)
5. 4.2.5.1 – NSF Policy position is inconsistent in stating that NSF has a “No Cost Overrun” policy
and that that Directorates are responsible for the first 10% of the cost overrun. Item 5 says “NSF
will hold budget contingency through project completion, in an amount up to 100% of the total
NSF-approved contingency budget, until it can be justified for obligation.” This provision may
skew the budget profile requested by a project towards higher award amounts in early years,
since the year-by-year funding must exceed the sum of expenditures, obligations, and risks in a
given year. A Project Manager, in dealing with the uncertainty in the level of availability of
budget contingency, may overestimate its potential need in early years. Ho will NSF protect
against this?
6. Section 4.5.3.2 External Reviews makes no distinction between operations and construction
reviews. It says: “The PO will typically use a standard review ‘template.’ These well-defined
review formats provide a broad outline against which the project can be compared and
checklists that can be used to assess the status of the project.” A footnote regarding the
template says: “Please contact the cognizant NSF PO for details and a description of best
practices and/or preferred templates.” To be effective, the Program Officer should have indepth professional knowledge of the scientific operation of the facility, and should develop a
review charge written to elicit advice matched to the specific needs and challenges of the

7.

8.

9.

10.

11.

12.
13.

14.

facility. The review should not be based on a check-the-box template. The PO should consult
with the DD, G/AO, DD, and LFO (where appropriate) to develop the charge.
4.2.2-9 Prohibited use of management fees. The NSF large facilities manual seems like a very
odd place to look for this information. Obviously NSF needs to make these policites clear to
awardees, but it might be better to direct the reader of the LFM, via a hyperling or reference, to
a more general source for information on this topic such as the CFR. That way the LFM will be
current if the CFR is updated.
4.5.3.1 misuses a lot of terms and creates confusion. It refers in a single sentence to conducting
reviews and internal audits in conformance with generally accepted accounting, project
management, and operations management standards and practices. Further in, it talks about
safety reviews, as well as many technically oriented reviews. Financial reviews of awardees, and
the internal audit and processes of the awardee, is a huge subject that should not be addressed
here. This guidance is useless and amateurish. Safety reviews of the awardee organization are
similarly the subject of state regulation, OSHA, and other standards. Often, awardees are state
universities subject to regulation and oversight. It seems absurd to include this in the PEP.
4.5.3.2 – LFO seems to have invited themselves to all operations reviews and assumed the right
to concur on all operational review charges. It claims the PO will use a standard review
template. It claims that exceptional circumstances may require the PO to consult with LFO
Liaison to constitute a review charge and format to meet the specific requirements of the
review. This is absurd – the NSF’s PO - scientific expert that follows the facility closely needs to
consult the LFO Liaison, who apparently has the time to participate in more than 20 reviews per
year, to develop specific requirements!
4.5.5 Rebaselining. Every re-allocation of budget or schedule contingency changes the baseline,
as does the use of scope contingency! Usually rebaselining occurs when the underlying planning
assumptions are so different from project reality that the major replanning is needed. What
criteria trigger the re-baselining definition? What should be done as a result? This section has
little information content, and needs to be rewritten to provide substantive definition.
4.2.5.8 – monthly reporting is a tricky issue – it should be done monthly but the paper work
reduction act says we cannot compel an awardee to do so. This section entirely omits reporting
technical status! It’s only financial status! It is very important that the technical status be
reported, accompanied by a narrative of progress relative to the plan and the factors that
contribute to cost and schedule variances. The liens should be listed by date for a decision on
whether to initiate a CCB action. A time-phased risk profile is helpful.
Heading 5.6 has a typo.
4.2.6 “Budget Contingency Planning during the Operations Stage” needs a better title to avoid
the implication that it is about including budget contingency in operation. It includes a
paragraph about divestment that would benefit from its own heading.
There is no discussion of performance incentives in the discussion of management fee. This has
been used by NSF at times.

LARGE FACILITIES MANUAL

Prepared by the Large Facilities Office in the Budget,
Finance, and Award Management Office (BFA-LFO)

Draft for Public Comment
NSF 16-XXX
May 2016

Large Facilities Manual: NSF 16-0XX (May 2016)
Summary of Significant Changes
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

SUMMARY OF SIGNIFICANT CHANGES
This document has been cleared for publication under OMB approval number 1315-0239, which
expires 6/30/2018. The purpose of this revision is to update the material and to improve the
clarity and legibility of the manual for the targeted audience of users both inside and outside
NSF. A summary of the changes for this revision is given below. Footers within each section of
this document indicate the last revision date of the content in that section, while all page
headers in the document include the date and NSF number of the current version of the Large
Facility Manual.
1.

Replaced the title of the “Deputy Director for Large Facility Projects, (DDLFP)” with the
new title” Head, Large Facilities Office (HLFO)” throughout the document. Created new
heading in Section 2.1.6, with the roles and responsibilities of the DDLFP listed as
"Reserved”. Changed issuance of the HLFO MREFC facility status report from “monthly”
to “periodic” on page 2.1.6-13.

2.

Revised Section 2.1.6 to clarify the roles of the Grants and Awards Officer and
Contracting Officer and to include the roles and responsibilities of the NSF Cost Analyst.
Four figures were added to illustrating the relationships and position within the NSF
organization of entities with roles and responsibilities for oversight and management of
large facilities.

3.

Removed references to “contingency reserves” throughout the document in order to
avoid confusion/conflict of the NSF definition of allowable “contingency” with the
definition of the term “reserves” contained in 200.433(c) of the Uniform Guidance, the
Project Management Book of Knowledge (PMBOK), GAO Cost Estimating and Assessment
Guide, or other publications.

4.

Renamed the last stage in the project lifecycle from “Termination Stage” to “Divestment
Stage” throughout the document.

5.

Language in Section 4.2.2.2 first paragraph was changed to clarify an illustrative example
of a circumstance appropriate for use of management fee. Also changed wording in the
third bullet under “Prior Approval of Management Fees” section on page 2.2.2-9, to
remove undue emphasis on the term “necessary” in discussions of “ordinary and
necessary” expenses.

6.

The NSF policy on withholding allocations of budget contingency for construction
projects until justified was added to sub-Section 4.2.5.1, NSF Policy Positions.

7.

Removed the bullet on page 4.2.5-4 referencing use of budget request form NSF 424C in
lieu of SF 1030, as this form is not used for large facility awards and the guidance is
overtaken by strategies in revised Sections 4.2.1 – 4.2.4.

Section Revision:
May 2, 2016
i

Large Facilities Manual: NSF 16-0XX (May 2016)
Summary of Significant Changes
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

8.

Section 4.5.3 was reorganized into sub-sections to clearly separate the types of
performance reviews. Existing text on recipient Internal Reviews (4.5.3.1), External
Reviews (4.5.3.2), and Business Systems Reviews (4.5.3.3) were revised for clarity.

9.

New material was introduced in a new sub-section, 4.5.3.4, Cost Incurred Audits,
including NSF requirements for recipients of award amounts >$100M to submit incurred
cost data on at least an annual basis using a mandatory Excel data tool created by NSF for
the purpose.

10. Section 4.5.3.5, Earned Value Management Validation, was created and reserved for
future content on the NSF requirements and process for validating construction project
EVM systems.
11. Language in Section 4.5.4, NSF’s Performance Measurement Metric, was re-written to
clarify that the metric is applied to total project variance against the performance
baseline.
12. Section 4.2.5.8, Reporting Requirements, was edited to clarify requirements for reporting
contingency status, and Section 4.5.2 Frequency and Content of Reports was edited to
include all reporting requirements. Redundancies were removed and references inserted
to improve clarity.
13. Section 5.5, Environmental Considerations for Large Facilities, was edited for clarity
regarding the Program Officers responsibilities and to include references to publically
available regulations. A NEPA acronym, “CATEX,” was added to the Section 7, Acronym
List.
14. Replaced the term “awardee” with the term ‘recipient” throughout the manual to allow
generic use on all award instruments (cooperative agreements, contracts, etc.).
15. Deleted text referencing the Major Overhaul Stabilization Account (MOSA) as an example
for operating budget development in Section 4.2.6.
16. Table 3.4.1-1, List of the Essential Components of a Project Execution Plan, was edited to
update language regarding cost, schedule, and scope estimating and management plans
definitions and terminology to be consistent with revisions and clarifications in Section
4.2, Cost Estimating and Analysis.
17. In Section 4.2, Cost Estimating and Analysis, major revisions were made to sub-Sections
2.4.1, 4.2.2, 4.2.3, and 4.2.4 to consolidate and clarify NSF guidance and requirements
and to ensure alignment with GAO guidelines. The material on Management Fees was
moved from the previous Section 4.2.2.2 to the new Section 4.2.2.5. The section on
Education and Outreach Budgeting during Operations (Section 4.2.4), was eliminated and
the material was incorporated into the new Section 4.2.4, Cost Estimating and Analysis
for Operations Awards.
Section Revision:
May 2, 2016
ii

Large Facilities Manual: NSF 16-0XX (May 2016)
Summary of Significant Changes
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

18. The material in Section 5.6, Guidelines for Reporting Requirements, was eliminated since
the material is now covered in Section 4.5, Requirements for Performance Oversight,
Reviews, Audits, and Reporting. Section 5.6 is now a place holder for future content on
Property Management.
19. References to a Financial Management Module in Section 5.7 and elsewhere in the
manual were eliminated as the document in question is unpublished and out of date.
Section 5.7 has been retained as a place holder for potential future content on Financial
Management.
20. Section 5.8, Guidelines for Conducting Business Systems Reviews of NSF Large Facilities,
was eliminated since the material was redundant with Section 4.5.3.3.
21. Section 5.9, OMB Inflators, was eliminated as the material was incorporated into the new
cost estimating Section 4.2.2.6.
22. New cost related acronyms (3) and terms (5) introduced in Section 4.2 revisions were
added to Section 7, List of Acronyms, and Section 8, Lexicon, respectively. The Acronym
List was also updated to include additional acronyms for roles within the NSF Office of
Budget, Finances, and Awards.
23. Corrections were made to various minor typographical errors and formatting errors.
Document references were updated to latest versions where applicable.

Section Revision:
May 2, 2016
iii

Large Facilities Manual: NSF 16-0XX (May 2016)
Table of Contents
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

TABLE OF CONTENTS
1 INTRODUCTION
1.1 Purpose and Scope
1.2 Precedence
1.3 Document Structure
2 LARGE FACILITY LIFE CYCLE AND THE MREFC PROCESS
2.1 MREFC Process Introduction
2.1.1 Definition of the MREFC Account
2.1.2 Eligibility for MREFC Funding
2.1.3 Facility Life Cycle
2.1.4 Summary of the MREFC Process
2.1.5 Timeline and Flowcharts for the MREFC Approval Process
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large
Facilities
2.1.6.1 Overview
2.1.6.2 Main Participants
2.1.6.3 Coordinating and Advisory Bodies
2.1.6.4 Governing Bodies
2.2 Development Stage
2.2.1 Initiation of a Potential MREFC Project
2.2.2 Exit from Development to Design Stage
2.3 Design Stage – Conceptual, Preliminary, and Final Design Phases
2.3.1 Conceptual Design Phase
2.3.1.1 Introduction – Conceptual Design Phase
2.3.1.2 Conceptual Design Phase Activities
2.3.1.3 Conceptual Design Review (CDR)
2.3.1.4 Exit from the Conceptual Design Phase
2.3.2 Preliminary Design Phase
2.3.2.1 Introduction – Preliminary Design Phase
2.3.2.2 Preliminary Design Phase Activities
2.3.2.3 Preliminary Design Review (PDR)
2.3.2.4 Exit from Preliminary Design Phase
2.3.2.5 NSF Director’s Recommendation for Advancement to Final Design
2.3.2.6 National Science Board Approval
2.3.2.7 Inclusion in an NSF Budget Request
2.3.3 Final Design Phase
2.3.3.1 Introduction – Final Design Phase
2.3.3.2 Final Design Review (FDR)
2.3.3.3 Exit from the Final Design Phase
2.4 Construction Stage
2.4.1 Construction Award Management and Oversight
Section Revision:
May 2, 2016
iv

Large Facilities Manual: NSF 16-0XX (May 2016)
Table of Contents
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.4.2 Commissioning Plan
2.4.3 Construction Award Close-out
2.4.3.1 Project Close-out Process
2.4.3.2 Request for No-Cost Extension
2.5 Operations Stage
2.5.1 Operations Management and Oversight
2.5.2 Renewal/Recompetition
2.6 Divestment Stage
2.7 Application of MREFC Process to Non-MREFC Funded Projects
2.7.1 Flexible Requirements for Non-MREFC Facility Projects
2.7.2 Pre-construction Planning and Development of Non-MREFC Projects
3 LIFE CYCLE MANAGEMENT PLANS FOR LARGE FACILITIES
3.1 Introduction to Management Plans
3.2 NSF Facility Plan
3.3 Internal Management Plans for the Large Facility Life cycle
3.4 Project Execution Plan
3.4.1 Components of a Construction-ready Project Execution Plan
3.4.2 Detailed Guidelines for Project Execution Plans
3.5 Operations Plan
3.5.1 Preparation of Proposals for Operations and Maintenance
3.5.2 Procedures for Renewal or Recompetition of an Operating Large Facility
3.5.3 Detailed Guidelines for Oversight of Operations
3.6 Facility Divestment Plan
4 KEY MANAGEMENT PRINCIPLES AND REQUIREMENTS FOR LARGE FACILITIES
4.1 Introduction
4.2 Cost Estimating and Analysis
4.2.1 Cost Estimating and Analysis Overview
4.2.2 Cost Estimating and Analysis Process
4.2.2.1 Cost Estimating Plan
4.2.2.2 Relevant Guidance and Reporting Formats
4.2.2.3 Funding Sources and Award Instruments
4.2.2.4 Supplementary Guidance for Construction and Operations Awards
4.2.2.5 Management Fees
4.2.2.6 OMB Inflators
4.2.3 Cost Estimating and Analysis for Construction Awards
4.2.3.1 Purpose and Process
4.2.3.2 Construction Cost Reports Overview
4.2.3.3 Construction Cost Book Report Format and Work Breakdown Structure
4.2.3.4 Construction Cost Book Detail and Supplementary Guidance
4.2.4 Cost Estimating and Analysis for Operations Awards
4.2.4.1 Purpose
4.2.4.2 Operations Awards Proposals Overview
Section Revision:
May 2, 2016
v

Large Facilities Manual: NSF 16-0XX (May 2016)
Table of Contents
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

4.3
4.4
4.5

4.6

4.2.4.3 Operations Awards Proposals Format
4.2.4.4 Operations Awards Proposals Supplementary Guidance
4.2.5 Budget Contingency Planning for the Construction Stage
4.2.5.1 NSF Policy Positions
4.2.5.2 Introduction
4.2.5.3 Contingency Planning and Assessment during Conceptual Design
4.2.5.4 Contingency Planning and Assessment during Preliminary Design
4.2.5.5 Development of the Contingency Use Process
4.2.5.6 Contingency Planning and Assessment during Final Design
4.2.5.7 Contingency Use and NSF Oversight during Construction
4.2.5.8 Reporting Requirements
4.2.5.9 Partnership Considerations
4.2.6 Budget Contingency Planning during the Operations Stage
System Integration, Commissioning, Testing and Acceptance
Documentation Requirements
Requirements for NSF Performance Oversight, Reviews, Audits, and Reporting
4.5.1 Introduction to Oversight, Reviews, Audits and Reporting
4.5.2 Recipient Performance Reports
4.5.3 Recipient Performance Reviews and Audits
4.5.3.1 Recipient Internal Reviews
4.5.3.2 NSF External Reviews
4.5.3.3 Business Systems Reviews (BSR)
4.5.3.4 Cost Incurred Audits
4.5.3.5 Earned Value Management Validation
4.5.4 NSF Performance Metric for Construction
4.5.5 Re-Baselining
Partnerships
4.6.1 Partnerships Overview
4.6.2 Partnership Funding
4.6.3 Memorandum of Understanding (MOU)

5 SPECIAL TOPICS AND SUPPLEMENTARY MATERIALS
5.1 Introduction
5.2 Risk Management Guidelines
5.2.1 Introduction
5.2.2 Definition of Project Risk and Risk Exposure
5.2.3 Definition of Allowable Contingencies
5.2.3.1 Allowable Contingency
5.2.3.2 Contingency Definitions
5.2.4 Risk Management Steps and Methodology
5.2.5 Risk Management Planning
5.2.5.1 Risk Management Plan (RMP)
5.2.5.2 Roles and Responsibilities
Section Revision:
May 2, 2016
vi

Large Facilities Manual: NSF 16-0XX (May 2016)
Table of Contents
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.3
5.4
5.5
5.6
5.7
5.8

5.2.6 Risk Identification
5.2.6.1 Risk Identification Process
5.2.6.2 Risk Identification and the Risk Register
5.2.6.3 Risk Description
5.2.6.4 Risk Identification Concerns
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
5.2.7.1 Purpose of Qualitative Risk Analysis
5.2.7.2 Considerations When Performing Qualitative Risk Analysis
5.2.7.3 Limitations of Qualitative Analysis
5.2.7.4 Qualitative Risk Analysis – Probability and Impact Assessment
5.2.7.5 Alternative Approach to Qualitative Risk Impact Analysis ‒ Maxwell
5.2.7.6 Qualitative Risk Analysis – Risk Level Matrix
5.2.7.7 Risk Level Input to the Risk Register
5.2.7.8 Other Qualitative Risk Analysis Methods
5.2.8 Quantitative Risk Analysis – Estimating Contingency
5.2.8.1 The Purpose of Quantitative Risk Analysis
5.2.8.2 Key Elements in Quantitative Risk Analysis
5.2.8.3 Platforms for a Project Quantitative Risk Analysis
5.2.8.4 Case Study: Quantitative Risk Analysis Exercise
5.2.8.5 Schedule Risk Analysis ‒ Uncertainty
5.2.8.6 Schedule Risk Analysis ‒ Discrete Risks Added as Drivers
5.2.8.7 Cost Risk Analysis ‒ Uncertainty and Discrete Risk Drivers
5.2.8.8 Handling Inflation
5.2.8.9 Prioritizing the Discrete Risks ‒ Risk Mitigation Workshop
5.2.9 Risk Response Planning
5.2.10 Risk Monitoring and Control
5.2.11 Contingency Management for Risk Mitigation
5.2.11.1 Contingency Budget Timeline
5.2.11.2 Change Control for Contingency Adjustments
5.2.11.3 Liens List: Forecasting and Opportunity Management
5.2.11.4 Updates of the Estimate at Complete and Risk Exposure
5.2.11.5 Contingency Use and NSF Oversight during Construction
5.2.11.6 Documentation and Reporting of Contingency Use
5.2.12 Partnership Considerations for Contingency Management
Guidelines for Cyber-Security of NSF’s Large Facilities
Guidelines for Planning and Executing External Reviews of NSF's Large Facilities
Environmental Considerations in Large Facility Planning
Guidelines for Property Mangement
Guidelines for Financial Management
Guidelines for Earned Value Mangement Systems

Section Revision:
May 2, 2016
vii

Large Facilities Manual: NSF 16-0XX (May 2016)
Table of Contents
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

6 REFERENCES
7 LIST OF ACRONYMS
8 LEXICON
8.1 Lexicon Preface
8.2 Terms and Definitions
9 APPENDICES
Appendix A.
Appendix B.

Ranking Criteria for Prioritizing MREFC Projects
Memoranda Requesting NSB Approval of MREFC Project

Section Revision:
May 2, 2016
viii

Large Facilities Manual: NSF 16-0XX (May 2016)
List of Figures
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

LIST OF FIGURES
Figure 2.1.3-1

Figure 2.1.3-2

Figure 2.1.4-1
Figure 2.1.6-1
Figure 2.1.6-2
Figure 2.1.6-3
Figure 2.1.6-4

Figure 4.2.1-1
Figure 4.2.1-2
Figure 4.2.2-1
Figure 4.2.2-2
Figure 4.2.3-1
Figure 4.2.3-2
Figure 4.2.4-1
Figure 4.2.5-1
Figure 4.5.3-1

Figure 5.2.4-1
Figure 5.2.6-1
Figure 5.2.6-2
Figure 5.2.6-3
Figure 5.2.7-1

Progressive Steps in the MREFC Life Cycle, Showing Review and Decision
Points for Exit and Entry into Each Stage. The Design Stage is highlighted
to indicate that it is further broken down into phases
Progressive Phases in the Life Cycle Design Stage, showing review and
decision points for advancement to the next phase and NSB approvals for
budgeting and award.
Summary Timeline for MREFC Projects
NSF organization chart highlighting staff who have primary oversight and
management roles and responsibilities for large facilities.
NSF organization chart showing coordinating and advisory bodies for
large facilities.
NSF organization chart showing planning, policy, and advisory bodies for
large facilities.
An Integrated Project Team (IPT), chaired by the Program Officer, is
composed of three subgroups, with appointed Award Management
Group members from BFA, Science and Technology Group members from
the originating program offices, and Strategic Group members from the
Office of the Director.
Cost Analysis Process During Design Phases for Construction Awards (Post
– CDR, PDR, FDR to Support MREFC Stage-Gate Review Process)
Cost Analysis Process for Operations Awards
Sample Project Management Control Systems Flow Chart
NSF Budget Categories Sample Format
Construction WBS and Cost Book Report Sample Format
Construction Cost Book Sheet Sample Format
Operations WBS and Budget Sample Format (For Each Physical Site
Location or Decentralized Overall Research Activity)
Sample of a Change Control Request Form, with instructions for filling out
the various sections
Large Facility Financial Data Collection Tool Process Flow Diagram,
identifying the core data sources recipients should utilize to complete the
worksheets to accurately capture the data. Progressing through the tool's
worksheets from left to right will streamline the data capture and
submission process.
Picture of Six Risk Management Processes (According to PMI)
Risk Identification Process
Typical Risk Breakdown Structure (RBS)
OMB Risk Categories: to be used as a starting point for projects to select
their own categories
Qualitative Risk Analysis Process

Section Revision:
May 2, 2016
ix

Large Facilities Manual: NSF 16-0XX (May 2016)
List of Figures
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Figure 5.2.7-2
Figure 5.2.7-3
Figure 5.2.7-4
Figure 5.2.7-5
Figure 5.2.7-6

Figure 5.2.7-7

Figure 5.2.8-1
Figure 5.2.8-2
Figure 5.2.8-3
Figure 5.2.8-4

Figure 5.2.8-5
Figure 5.2.8-6
Figure 5.2.8-7
Figure 5.2.8-8
Figure 5.2.8-9
Figure 5.2.8-10
Figure 5.2.8-11
Figure 5.2.8-12
Figure 5.2.8-13
Figure 5.2.9-1
Figure 5.2.10-1

Overlap in Risk Probability of Occurring When Descriptors Are Used
Symmetrical Risk Level Matrix
Asymmetrical Risk Level Matrix
Probability and Impact Matrix including Threats and Opportunities
Sample Risk Register with Risk ID Number, Associated WBS Identification,
Qualitative Probability and Impact for Initial and Post-mitigation States,
and Mitigation Actions
Sample Top Risk Matrix and Status Report, showing list of project risks
selected as most significant to monitor on a frequent basis, with ranking
and trend data
Quantitative Risk Analysis Process
Typical Result of a Quantitative Schedule Risk Analysis
Time-Cost Scatter Diagram: Each data point represents one realization of
the simulation
Resource Loaded Schedule Used for a Simple Case Study of an Integrated
Cost-Schedule Risk Analysis for Design, Fabrication, Testing, and Delivery
of a Space Vehicle
End Date Results for Schedule Duration Uncertainties
Cost Result for Schedule Duration Uncertainties
Total Cost and End Date Scatterplot for Schedule Uncertainties
Schedule Risk Drivers – Organizational Risk
Total Cost and End Date Scatterplot Showing Greater Correlation of Time
and Cost Risk
Uncertainty in the Burn Rate and Total Cost
Screenshot of Risk Driver Editor
Scatterplot Showing Less Connection between Time and Cost
Scatterplot with Addition of Cost Inflation Factor
Risk Response Planning Process
Risk Monitoring and Control Process

Section Revision:
May 2, 2016
x

Large Facilities Manual: NSF 16-0XX (May 2016)
List of Tables
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

LIST OF TABLES
Table 2.1.6-1
Table 2.4.3-1
Table 3.4.1-1
Table 5.2.5-1
Table 5.2.5-2
Table 5.2.7-1
Table 5.2.7-2
Table 5.2.7-3
Table 5.2.8-1
Table 5.2.8-2
Table 5.2.8-3
Table 5.2.8-4
Table 5.2.8-5
Table 5.2.8-6
Table 5.2.9-1

Summary of Principal Roles and Responsibilities of the core members of
the IPT (PO, G/AO or CO, and LFO) Liaison by Facility Life Cycle Stage
Sample of a No-Cost Extension Tasks Table
List of the Essential Components of a Project Execution Plan, with SubTopics and Descriptions
Sample Format for a Risk Management Plan
Example of a Risk Management Roles and Responsibilities Table
Sample Risk Probabilities Table
Sample Risk Consequences Table
Maxwell Risk Driver Assessment Framework
Resources for Quantitative Risk Analysis Example
Schedule Duration Risk Reference Ranges
Results with Schedule Uncertainties Assigned
Results with Schedule Uncertainties and Discrete Risks Assigned
Results with Uncertainties and Cost Risks Assigned
Savings and Days Saved
Impact of Risk Handling on Project Cost

Section Revision:
May 2, 2016
xi

Large Facilities Manual: NSF 16-0XX (May 2016)
1.1 Purpose and Scope
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

1

INTRODUCTION

1.1

PURPOSE AND SCOPE

A major responsibility of the National Science Foundation (NSF) is the support of scientific
facilities as an essential part of science and engineering enterprise. NSF makes awards to
external Recipient entities – primarily universities, consortia of universities or non-profit
organizations – to undertake construction, management, and operation of facilities. Such
awards frequently take the form of cooperative agreements.1 Generally speaking, NSF does not
directly construct or operate the facilities it supports. However, NSF retains responsibility for
overseeing the Recipient’s development and management of the facility as well as assuring the
successful performance of the funded activities.
Facilities are defined as shared-use infrastructure, instrumentation and equipment that are
accessible to a broad community of researchers and/or educators. Facilities supported by NSF
may be centralized or may consist of distributed-but-integrated installations. They may
incorporate large-scale networking or computational infrastructure, multi-user instruments or
networks of such instruments, or other infrastructure, instrumentation, and equipment having
a major impact on a broad segment of a scientific or engineering discipline. Historically, awards
have been made for such diverse projects as accelerators, telescopes, research vessels, aircraft,
and geographically distributed but networked observatory systems.
The Large Facilities Manual contains NSF policy on the planning and management of large
facilities. The purpose of the Manual is to provide guidance for NSF staff and recipients to:




Carry out effective project planning, management, assistance, assurance, and oversight
of large facilities,
Clearly state the policies, requirements, and recommended procedures pertinent at
each stage of a facility’s life cycle, and
Document best practices that ensure accountability and effectiveness of the program.

The policies in the Large Facilities Manual apply to all large facility projects funded by NSF,
including:



Large facilities that have been or will be constructed or acquired with funds from the
Major Research Equipment and Facility Construction (MREFC) Account;
Facilities or infrastructure projects that have been or will be constructed or acquired
with funds provided through the Research and Related Activities (R&RA) and/or
leveraged with Education and Human Resources (EHR) Accounts and that require
National Science Board (NSB) approval; and

1

See NSF Proposal and Award Policies and Procedures Guide for detailed information on awards. NSF staff should also be
consulted for material contained in the internal NSF document, Proposal and Award Manual.

Section Revision:
June 17, 2015

1.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
1.1 Purpose and Scope
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)



Existing facilities for which operation and replacement cost would be similar in size to
MREFC-funded and MREFC-eligible projects.

NSF typically supports facility construction from two appropriations accounts: the MREFC
Account and the R&RA Account. The MREFC Account was created in 1995 to fund the
acquisition, construction, commissioning, and upgrading of major science and engineering
infrastructure projects that could not be otherwise supported by Directorate level budgets
without a severe negative impact on funded science. MREFC projects generally range in cost
from one hundred to several hundred million dollars expended over a multi-year period. The
R&RA account can be used to support other activities involving an MREFC-funded facility that
the MREFC Account cannot support, including planning, conceptual design, development,
operations and maintenance, and scientific research. Construction and acquisition projects at a
smaller scale, usually of a scale ranging from millions to tens of millions of dollars, are also
normally supported from the R&RA Account. The provisions and principles in the Large
Facilities Manual also apply to these smaller-scale facilities funded through the R&RA Account,
but procedures should be modified appropriately to fit the needs of each facility.

Section Revision:
June 17, 2015

1.1-2

Large Facilities Manual: NSF 16-0XX (May 2016)
1.2 Precedence
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

1.2

PRECEDENCE

The Large Facilities Manual comprises Chapter XIII of the Proposal and Award Manual (PAM),
but is published under separate title. It replaces the Large Facilities Manual, NSF 13-38,
published in 2013, and incorporates changes in organization and content intended to clarify the
policies and procedures by which MREFC candidate projects are identified, developed,
prioritized and selected.1
The Large Facilities Manual is a public document owned and managed by the Office of Budget,
Finance and Award Management’s (BFA) Large Facilities Office (LFO). It is available on the LFO
public website (https://www.nsf.gov/bfa/lfo/index.jsp) as well as through the internal LFO
website.
The Manual does not replace existing formal procedures required for all NSF awards, which are
described in the publically available Proposal and Award Policies and Procedures Guide and in
the NSF internal Proposal and Award Manual. Instead, it draws upon and supplements them for
the purpose of providing detailed guidance regarding NSF management and oversight of
facilities projects. All facilities projects require merit review, programmatic/technical review,
and a substantial approval process. This level of review and approval differs substantially from
standard grants, as does the level of oversight needed to ensure appropriate and proper
accountability for federal funds. The policies, requirements, recommended procedures, and
best practices presented herein apply to any facility large enough to require interaction with
the NSB or any facility so designated by the Director, the Deputy Director, or the Assistant
Director/Office Head of the Originating Organization(s).2 For all other facilities, NSF staff
members should use their judgment in proportionately scaling the requirements and
recommended procedures for specific projects.
This Manual will be updated periodically to reflect changes in requirements and/or policies.
Program Officers (PO) are encouraged and expected to continue to identify and adopt best
practices aimed at improving NSF oversight and Recipient management of large facilities
projects and at enabling the most efficient and cost-effective delivery of tools to the research
and education communities.

1

See the Joint National Science Board —National Science Foundation Management Report: Setting Priorities for Large Facility
Projects Supported by the National Science Foundation (NSB-05-77); September 2005

2

See Section 2.1.6 for definition of this and other key terms. It also describes the NSF organizations and officers that are
involved throughout the initiation, development, approval and implementation of an MREFC project. Readers not familiar with
NSF and its processes should review this material before proceeding.

Section Revision:
June 17, 2015

1.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
1.3 Document Structure
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

1.3

DOCUMENT STRUCTURE

The Manual is organized as follows:









Section 1 introduces the purpose, scope, and historical perspective of this document.
Section 2 describes the life cycle stages and the process and principles NSF uses to plan,
construct and operate large facilities. The steps for approval and execution of projects
funded using the MREFC Account and the roles and responsibilities of NSF staff are
detailed.
Section 3 describes the requirements for preparing and following the various detailed
management plans required during the life cycle of a large facility.
Section 4 is an expanded compendium of several NSF key requirements and principles
listed in Sections 2 and 3. It includes detailed descriptions of processes used to plan,
acquire, and manage large facilities.
Section 5 contains extensive supplementary information on specific topics concerning
NSF’s role in the planning, oversight, and assurance of large facility projects. It consists
of sections containing important explanatory and procedural information and pointers
to separate documents (or modules) with similar information. The information in the
documents is presented in a tutorial format that should be of particular benefit to
individuals who are newly involved with large facility projects.1
The appendices contain other information relevant to MREFC projects and large
facilities.

This Manual is intended for use by NSF staff and by external proponents of large facility projects
for use in planning. However, there are occasional references to materials, such as the NSF
Proposal and Award Manual2 (PAM), which are available only internally to NSF staff and refer to
details of NSF administrative practices and procedures that are not relevant to external project
proponents. Wherever these internal references are included, they are clearly noted as such.
Owing to the rigor of merit and programmatic review, constraints on funding, changing
priorities and competing interests of NSF and the research community, only a limited number
of projects will proceed successfully through all stages described herein. To improve the
possibility of success, facility advocates should be thoroughly familiar with the entire contents
of this manual even if the proposed project is in the earliest stages of formulation. Anticipating
downstream requirements will dramatically improve the efficiency of the process.

1

Section 5 will be further updated to include additional information, with the intent to provide to NSF and the research
communities a single reference location for all relevant policies and procedures.
2

The NSF Proposal and Award Manual is a compendium of internal policies and procedures.

Section Revision:
June 17, 2015

1.3-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1 MREFC Process Introduction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2
2.1

LARGE FACILITY LIFE CYCLE AND THE MREFC PROCESS
MREFC PROCESS INTRODUCTION

National Science Foundation (NSF) investments through the Major Research Equipment and
Facility Construction (MREFC) Account provide state-of-the-art infrastructure for research and
education, such as laboratory and field instrumentation and equipment, multi-user research
facilities, distributed instrumentation networks and arrays, and mobile research platforms. In
addition, investment is increasing in highly sophisticated information technology (IT)-based
infrastructure, including distributed sensor networks, extensive data-storage and transmission
capabilities, advanced computing resources, and Internet-based distributed user facilities.1
This section describes the overall MREFC process as well as the roles and responsibilities of the
various participants. It provides guidelines for planning and managing facilities supported
through the MREFC Account. Because each facility has unique aspects, each project necessarily
requires adaptation of general principles. NSF promotes flexibility in the application of these
guidelines, but requires justification and substantiation for the specific approach taken in each
case. That is accomplished through the processes of formal planning, documentation, and
review.

1

These resources, many of which are now in development, are collectively known as “cyber infrastructure.”

Section Revision:
June 17, 2015

2.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.1 Definition of the MREFC Account
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.1.1 Definition of the MREFC Account
The MREFC Account is an agency-wide account, created in 1995 with Congressional approval,
which provides funding to establish major science and engineering infrastructure projects that
exceed 10% of the Directorate’s annual budget, or roughly one hundred million dollars or
greater. Specifically, the MREFC Account is intended to:




Provide a special account to fund acquisition, construction and commissioning of major
facilities and other infrastructure projects;1
Prevent large periodic obligations from distorting the budgets of NSF Directorates and
program offices; and
Ensure availability of resources to complete large projects that are funded over several
years.2

The MREFC Account funding is specifically for construction related activities; it cannot be used
to support other activities involving an MREFC-funded facility, such as planning, conceptual
design, development, operations and maintenance, or scientific research.

1

In some cases, MREFC funds may be used to support development after construction of a facility begins.

2

Reliable long-term funding commitments are essential to maintaining partnerships and for preventing cost overruns due to
schedule delays.

Section Revision:
June 17, 2015

2.1.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.2 Eligibility for MREFC Funding
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.1.2 Eligibility for MREFC Funding
To be eligible for consideration for MREFC funding, each candidate project should represent an
outstanding opportunity to enable research and innovation, as well as education and broader
societal impacts. Each project should offer the possibility of transformative knowledge and the
potential to shift existing paradigms in scientific understanding, engineering processes and/or
infrastructure technology. Moreover, each should serve an urgent contemporary research and
education need that will persist for years beyond the often lengthy process of planning and
development.
In addition, a candidate project should:









1

Be consistent with the goals, strategies and priorities of the NSF Strategic Plan;1
Establish a long-term tools capability accessible to an appropriately broad community of
users on the basis of merit;
Require large investments for construction/ acquisition, over a limited period of time,
such that the project cannot be supported within one or more NSF Directorate(s)/
Office(s) without severe financial disruption of their portfolios of activities;
Have received strong endorsement of the appropriate science and engineering
communities, based upon a thorough external review, including an assessment of
(1) scientific and engineering research merit, (2) broader societal impacts,
(3) importance and priority within the relevant Science and Engineering communities,
(4) technical and engineering feasibility, and (5) management, cost, and schedule issues;
Be of sufficient importance that the Originating NSF Organization2 is prepared to fully
fund the costs of pre-construction planning, design and development, operation and
maintenance, and associated programmatic activities (with full awareness that, for a
long-lived facility, operations costs may ultimately amount to many times the
construction costs); and
Have been coordinated with other organizations, agencies and countries to ensure
complementarity and integration of objectives and potential opportunities for
collaboration and sharing of costs.

Empowering the Nation Through Discovery and Innovation: NSF Strategic Plan for Fiscal Years (FY) 2011-2016.

2

See Section 2.1.6 for definition of this and other key terms. It describes the NSF organizations and officers that are involved
throughout the conception, development, approval and implementation of an MREFC project.

Section Revision:
June 17, 2015

2.1.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.3 Facility Life Cycle
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.1.3 Facility Life Cycle
A facility’s lifetime is characterized by the following life cycle stages:
1.
2.
3.
4.
5.

Development1
Design
Construction
Operation
Divestment.

Each life cycle stage entails different actions appropriate to the advancement of the project, the
review and approval needed to obtain NSF funding, and the creation of NSF budgets to support
these activities. Entry and exit from each life cycle stage are clearly defined including required
documents and deliverables. A high-level graphic of the progression through the stages is given
below in Figure 2.1.3-1.
Figure 2.1.3-1

Progressive Steps in the MREFC Life Cycle, Showing Review and Decision Points for Exit and
Entry into Each Stage. The Design Stage is highlighted to indicate that it is further broken down
into phases

Development

Design

Design Phase
Reviews
(Program, MREFC Panel
& DRB, OD & NSB)

Construction

Annual
Construction
Reviews
(Program)

Operations

Divestment

Annual Operations
Reviews

Renewal &
Re-competition Reviews

(Program)

(Program, MREFC Panel & DRB,
OD & NSB)

= Review (Stage gate, annual construction, etc.)
= NSF Decision Point (Program, Director, NSB)

Points at which there may be departure from the MREFC process outlined here should be
identified early in the project development and documented as part of the NSF Internal
Management Plan (IMP) Individuals should discuss any proposed departures with the cognizant
Program Officer.
The Design Stage is further divided into three phases, each with defined entry and exit points,
as shown in Figure 2.1.3-2 below:
1. Conceptual Design
2. Preliminary Design
3. Final Design
1

A project in the Development Stage may be referred to as a “Horizon” or a Conceptual Development project in earlier NSF
documents and references.

Section Revision:
May 2, 2016

2.1.3-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.3 Facility Life Cycle
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Figure 2.1.3-2

Progressive Phases in the Life Cycle Design Stage, showing review and decision points for
advancement to the next phase and NSB approvals for budgeting and award.

Conceptual Design Phase

Preliminary Design Phase

Final Design Phase

Preliminary Design Review (PDR):

Final Design Review (FDR):

MREFC Panel & Director’s Review
Board (DRB) Reviews

MREFC Panel & DRB Reviews

MREFC Panel Review
OD Approval for Advancement
to Preliminary Design

Director’s approval for Advancement
to Final Design

Conceptual Design
Review (CDR):

Board approval for inclusion in
future MREFC Budget Request
(CPP/any meeting prior to August)

Director’s Approval for Advancement
to Construction
Board approval for the Director to
obligate construction funds

Project Definition Established
(Cost, Scope, Schedule,
Plans, Risks & Contingency)

Summary definitions of each stage and phase are given below. See sub-sections 2.2 to 2.6 for
detailed discussions of the various procedures and deliverables for each step in the facility life
cycle.
Development Stage
The life cycle stage in which initial ideas emerge and a broad consensus built for the potential
long-term needs, priorities, and general requirements for Research Infrastructure (RI) of
interest to NSF. Investments in development by NSF, other government agencies, or private
interests can be focused or sporadic but are generally smaller annual investments than in the
Design Stage. The effort is focused on the high-level ideas and building community consensus
on requirements and setting priorities across a broad landscape of potential needs. This stage
can last 10 years or more. The cumulative investment over this period can be quite substantial.
Next to the Divestment Stage, the Development Stage is often the most challenging to navigate
depending upon how federal agencies and science communities are organized. The exit process
from this stage begins, once the issues have coalesced, with a proposal from a Division (via its
Directorate) to the NSF Director that a project is ready to begin the Conceptual Design Phase.
At that point a formal senior-level agency internal review takes place and the NSF Director may
approve this transition to the first phase in the Design Stage, with the provision that no
commitment to advance the project beyond the approved design phase is implied.
Design Stage
This is the life cycle stage for detailed planning of RI which is formally approved by the NSF
Director and funded by the sponsoring Directorate or Division as a candidate MREFC (or other)

Section Revision:
May 2, 2016

2.1.3-2

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.3 Facility Life Cycle
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

project. It is divided into the Conceptual, Preliminary, and Final Design Phases; with a formal
and rigorous review gate at the end of each phase to show readiness for advancement to a
higher level of design readiness. Entrance into this stage occurs when the NSF Director
approves the proposed Research Infrastructure as a national priority and the Directorate
obligates funding to detailed cost, scope and schedule development for possible construction.
This stage includes a series of readiness reviews to assure proper design progress and
advancement through the defined phases. This Stage generally lasts 3-5 years and costs 10% or
more of the estimated construction cost depending on the nature of the infrastructure. It is also
the stage where construction funds are identified and (ideally) were partnerships are
formalized.
Conceptual Design Phase: Advances the definition of the scope and requirements,
determines feasibility, and produces updated drafts of most elements of the Project
Execution Plan, including parametric cost and schedule range estimates and a
preliminary risk analysis.
Preliminary Design Phase: Further advances the project baseline definition and the
Project Execution Plan. It produces a bottom-up scope, cost, schedule, and risk analysis
of sufficient maturity to allow determination of the Project Total Cost and overall
duration for a given Fiscal Year start and to establish the MREFC budget request to
congress.
Final Design Phase: Further refines the project baseline definition and the Project
Execution Plan and demonstrates that project planning and management meet
requirements for readiness to receive funding. The Final Design phase ends after review
and NSF recommendation to approve the obligation of construction funds.
Construction Stage
This stage begins when MREFC funds are obligated for acquisition and/or construction of RI that
fulfills the terms and conditions set forth in an award instrument between NSF and the
recipient(s). Depending on the nature and scale of the facility, construction typically lasts 2-6
years and costs between $100M and $800M. This stage has the most stringent requirements
for managing the scope, cost, schedule, and performance; for reporting progress; and for
formality of oversight and assurance by NSF. Progress is reported against the approved Project
Execution Plan and project status is reviewed periodically to ensure that the project is capable
of finishing within budget and schedule. The Construction Stage normally includes activities to
transition to operations. It ends after final delivery and acceptance of the defined scope of work
and facility performance per the terms of the award agreement.
Operations Stage
This life cycle stage includes the day-to-day work to operate and maintain the RI and to perform
research. During this stage, the facility is actively collecting and distributing data for use by the
Section Revision:
May 2, 2016

2.1.3-3

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.3 Facility Life Cycle
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

science community. Operations may include activities to transition from construction to
operations, refurbishment or upgrade activities, and activities that support planning for and
transition to the Divestment Stage. This stage typically lasts 20-40 years, the total cost of which
often greatly exceeds the cost of construction. It normally includes a series of periodic status
reviews but may also include reviews and decisions on further investment, capability up-grades,
and refurbishments and eventually the final decision on divestment. Annual operating costs
and Concept of Operations Plans (including operational agreements between parties for
funding, data sharing, etc.) should already be well established before entering this stage. The
decision to divest is generally made when NSF and/or the scientific community determine that
the facility is no longer considered an operational priority with regard to advancing science. This
final decision is often the most challenging.
Divestment Stage
Entrance into this stage occurs when the first financial investment is made to divest or
decommission the RI. The decision to divest happens at the end of the Operations Stage.
Divestment could include transfer to another entity’s operational and financial control or
decommissioning, including complete de-construction and removal of the infrastructure. Cost
of decommissioning can be substantial and must be thoroughly considered in terms of process
and costs as part of divestment planning.

Section Revision:
May 2, 2016

2.1.3-4

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.4 Summary of the MREFC Process
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.1.4 Summary of the MREFC Process
MREFC projects cover a wide range of disciplines and activities in science and engineering, and
they can require rather different approaches to the development and ultimate acquisition of
facilities, equipment, and/or infrastructure. The approach described in this Manual is derived
largely from experience with construction projects defined by the following characteristics:




They serve a relatively large community or a large collaboration, whose members have
organized and agree on the basic parameters of the project; and
They result from proposals to NSF, either unsolicited or through a targeted NSF
solicitation, proposing to construct the particular equipment or infrastructure; and
Operation of the equipment or infrastructure is carried out by the entity or community
that proposes construction (or by some other entity in cases where the operations
expertise may not necessarily reside with the construction team).

As the diagram in Figure 2.1.4-1 indicates, pre-construction planning and development for
MREFC candidate projects progress through a sequence of stages of increasing investment,
planning, assessment, oversight, and assurance. Among other uses, these stages ensure that
the technical evolution of a candidate project is coordinated with NSF requirements, thus
increasing the likelihood that it will be able to qualify for funding for further planning and
eventual construction.
However, because NSF supports investigation at the frontiers of understanding, where specific
research targets and methodology often are not firmly established, some candidate projects
may need to progress in ways that are not as neatly well-defined as the prototypical cases
described above. The guidelines in this Manual allow for such cases. For example:




Because NSF is responsible for nurturing the various science and engineering disciplines
that it supports, it may provide researchers access to funding sufficient to develop
compelling research agendas, to refine and prioritize their facility requirements, and to
complete research and development on facility designs and needed technologies,
without assuming a direct role in overseeing either construction or operation. Such
projects should nevertheless be sanctioned by, and ultimately driven by, the community
through merit review that establishes that candidate new facilities represent a high
priority of the researchers in that discipline.
Following successful concept development, the entire project may be best developed
and implemented by an award directly to industry – for example, in the case of cyberinfrastructure. In such cases, provision should nevertheless be made for proper preconstruction planning, with thorough community input and merit review, followed by
proper oversight throughout the implementation stage of the project.

In all cases, NSF is committed to the principle that flexibility does not preclude rigor. Every
MREFC candidate project – including those that call for novel treatment – is subject to the
highest standards of merit review and technical evaluation.
Section Revision:
May 2, 2016

2.1.4-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.4 Summary of the MREFC Process
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 2.1.4-1

Summary Timeline for MREFC Projects

Project evolution

Budget evolution

FIGURE 1
Conceptual Design Phase

Preliminary Design
(Readiness) Phase

Final Design (Board
Approved) Phase

Construction

Preconstruction Planning Funded via R&RA and EHR funds
Develop construction budget
based on conceptual design
Develop budget requirements for
advanced planning

MREFC funds

Operations

R&RA, EHR funds

Expend ~5-25% of construction cost on planning & design activities
Construction estimate based on
prelim design
Update ops $ estimate

Estimate ops $

Final design over ~ 2 years
Construction-ready budget &
contingency estimates

Expend budget &
contingency per baseline

Yearly budgets with outyear projects

Refine ops budget

Update ops $ estimate

Proponents development strategy defined in Project Development Plan
Formulate science goals; define
requirements, prioritize, review

Develop site-specific preliminary
design, environmental impacts

Develop final constructionready design & PEP

Develop conceptual design; identify
critical technologies, high risk items

Develop enabling technologies

Industrialize key technologies

Formulate initial risk assessment

Bottom-up cost and contingency
estimates, updated risk analysis

Refine bottom-up cost and
contingency estimates

Develop top-down parametric cost
and contingency estimates

Develop Project Management
Control System

Finalize Risk Assessment &
Mitigation, Management Plans

Initial proposal submission to NSF

Develop preliminary operations
cost estimate

Complete key staff recruitment

Initial Project Execution Plan (PEP)

Construction
per baseline
and PEP

Annual Work Plans with
goal setting
Annual Reports that track
progress relative to goals

Update PEP

MREFC Panel recommendation
 Director approval for CD start
Develop Internal Management
Plan (IMP), est. PD costs, timeline
Establish interim review plan and
competition milestones

Forecast international and
interagency participation, issues
Initial analysis of NSF
opportunities, risks
Conceptual Design Review
(CDR)

Section Revision:
May 2, 2016

NSF Director approves PD
start, Internal Mgmt Plan (IMP)
Approve Project Development
Plan (PDP) & budget
Evaluate design costs,
schedules; and ops costs est.
Forecast external partner
decision milestones

Preliminary Design Review
(PDR) & integrated baseline
review, cap total project budget
NSF Director requests NSB
approval for MREFC request
NSB prioritization

OMB/Congress
negotiations on proposed
project and budget profile
Semi-annual assessment
of baseline and projected
ops budget for projects not
in construction
Finalization of interagency
and international
requirements, agreements
Final Design Review
(FDR), fix baseline

Congress appropriates
MREFC funds & NSB
approves obligation

Congress appropriates funds

and

NSF approves submission to NSB

Merit review, apply
ranking criteria

2nd

NSF Director approves PD phase

Program & Oversight evolution

NSF oversight defined in Internal Management Plan, updated by development phase

1st

Periodic external review
during construction

Annual or regular
reviews of operations
Preparation for facility
re-competition, renewal
or disposal/divestment

Review of project
reporting
Site visit and
assessment

2.1.4-2

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.5 Timeline and Flowcharts for the MREFC Approval Process
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.1.5 Timeline and Flowcharts for the MREFC Approval Process
This section, to be written, will illustrate when various preconstruction planning activities
should be completed in order to commence construction in a particular future fiscal year.
Although the majority of those activities proceed at a pace specific to the needs of an individual
project, late-stage planning activities following completion of a project’s Preliminary Design are
paced by the process for developing NSF’s annual Budget Request to Congress. This section will
also explain key features of that process that are of particular interest to those involved with
MREFC projects.

Section Revision:
June 17, 2015

2.1.5-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large
Facilities
2.1.6.1 Overview
The Large Facilities Manual (LFM) describes the actions NSF takes to carry out its oversight and
assurance responsibilities for large facility projects. One key element is the definition of the
roles and responsibilities of the NSF participants who carry out those actions. The participants
with primary oversight and management roles and responsibilities are listed below and
highlighted in the NSF organizational chart in Figure 2.1.6-1:














Program Officer (PO) – A scientist or engineer having primary oversight responsibility
within NSF for all aspects of the project.1
Originating Organization – The NSF Division, Directorate, or Office which proposes
projects for funding through the MREFC Account or other funding source and is
committed to pre-construction planning activities and eventual facility operation and
use.
Senior Management of the Originating Organization – The leadership individuals who
utilize community inputs, discipline-specific studies, advisory committee
recommendations and internal NSF considerations to prioritize the opportunities
represented by the candidate project relative to competing opportunities and demands
for available resources.
Grants and Agreements Officer or Contracting Officer (G/AO) – NSF grants
management officer who has legal responsibility and authority for the business and
financial management of grants and cooperative agreements.
Contracting Officer (CO) - NSF contracts management staff which have legal
responsibility and authority for the business and financial management of award
contracts.
Cost Analyst – NSF staff from the Cost Analysis and Audit Resolution (CAAR) Branch of
the Division of Institution and Award Support (DIAS), which perform cost assurance
reviews of proposals and monitor awardee financial practices.
Deputy Director for Large Facility Projects, (DDLFP) – [Reserved]
Head, Large Facilities Office (HLFO), – The individual who heads the Large Facilities
Office (LFO). The LFO provides an NSF-wide resource for assistance with project
oversight and assurance. The LFO is administratively located in BFA.
Large Facilities Office Liaison – The designated project management advisor from the
LFO, who is assigned as project liaison by the HLFO. This individual is the PO’s primary
resource for assistance with all policy, process, and procedural issues related to the
development, implementation, and oversight of MREFC projects.

1

The PO may have a title such as Program Manager or Program Director. The PO is administratively part of a Directorate or
Office, comprised of Divisions, which serves a range of research disciplines. These are referred to as the “originating Division,
Directorate or Office” in this document.

Section Revision:
May 2, 2016

2.1.6-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Figure 2.1.6-1

NSF organization chart highlighting staff who have primary oversight and management roles
and responsibilities for large facilities.

As shown in Figure 2.1.6-2, various bodies within NSF provide coordination, assistance,
assurance, and advice to the main participants and to the agency as a whole:




Integrated Project Team (IPT)1 – Comprised of NSF personnel with knowledge and
expertise in areas related to the scientific and technical, award management, and
strategic aspects of a particular MREFC project. The IPT is a coordinating body that
provides internal agency assurance and guidance to the PO in the planning, review, and
oversight of that project. The members of the IPT are selected by the management of
the cognizant directorates and offices, in consultation with the PO, at the beginning of
the Conceptual Design Phase. The IPT is chaired by the PO.
Large Facilities Working Group (LFWG)2 – An advisory group composed of internal NSF
staff and administrators that reviews and provides feedback to the LFO on draft policies,
processes and procedures related to NSF Large Facilities (including but not limited to the
Large Facilities Manual); raises and provides inputs on issues related to NSF Large
Facilities; and reviews and comments on IMPs prior to the Conceptual Design Review
(CDR).

1

The IPT replaces the Program Advisory Team (PAT) and the Business Oversight Team (BOT).

2

The LFWG replaces the Facilities Panel.

Section Revision:
May 2, 2016

2.1.6-2

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)



Advisory Committee of the Originating Directorate or Office – Comprised of
researchers from the community (external to NSF), it advises the originating Directorate
or Office in a wide variety of programmatic areas, including large facilities.

Figure 2.1.6-2

NSF organization chart showing coordinating and advisory bodies for large facilities.

There are also planning and assurance bodies, shown in Figure 2.1.6-3, that review and make
recommendations on the suitability and readiness as well as on the allocation of resources for
the development, funding, and operation of large facility projects, according to the NSF
strategic objectives:




MREFC Panel – Comprised of Senior Management representatives from the Directorates
and Offices of NSF, it reviews and recommends projects for advancement through the
MREFC process, as presented by the Originating Organization(s), and makes
recommendations to the NSF Director on priorities and use of NSF resources.
Director’s Review Board – Comprised of Senior Management Representatives from the
Directorates and Offices of NSF, it reviews and approves the package of materials
associated with all topics to be submitted to the National Science Board (NSB) for
information or action, including MREFC projects.

Section Revision:
May 2, 2016

2.1.6-3

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Finally, there are entities also shown in Figure 2.1.6-3 that set NSF policy and that approve the
advancement, funding requests, and obligation of funds for the development, construction, and
operation of large facility projects:



NSF Director – Has ultimate responsibility for the approval of the obligation of funds
from the MREFC Account and for proposing new MREFC projects to the NSB, the Office
of Management and Budget (OMB), and Congress.
NSB – Establishes policy, reviews and approves MREFC Account budgets, and reviews
and approves specific MREFC projects for funding.

Figure 2.1.6-3

NSF organization chart showing planning, policy, and advisory bodies for large facilities.

The PO, G/AO, and LFO staff members are the individuals that interact most frequently to carry
out NSF’s oversight and assurance role for large facility projects. Their roles and responsibilities
are summarized, by life cycle stage, in Table 2.1.6-1. Fuller descriptions of their roles (and those
of senior management in the originating Directorate or Office, and the support, advisory, policy
making, and approving entities) are provided in individual sections of this document following
Table 2.1.6-1.

Section Revision:
May 2, 2016

2.1.6-4

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 2.1.6-1

Summary of Principal Roles and Responsibilities of the core members of the IPT (PO, G/AO or
CO, and LFO) Liaison by Facility Life Cycle Stage

Program Officer (PO)

Grants/Agreements (G/AO) or
Contracts Officer (CO)

LFO Liaison

 Primary representative of the
NSF in all business dealings
with the recipient
 Assigned to a project on a
long-term basis
 Experienced with Federal
regulations and unique NSF
requirements needed for
adequate NSF oversight of
large facility projects

 Program’s primary resource
for all policy or process
issues related to the
development,
implementation, and
oversight of MREFC projects
 Advises POs on project
management issues during
project development and
oversight

 Becomes acquainted with the
anticipated scope of the
proposed project
 Participates in planning
meetings to work out details
of partnerships, international
or multi-agency agreements,
property issues, etc.
 Participates in the
development of the IMP
 Serves on the IPT throughout
the project

 In collaboration with PO,
plans CDR
 Independently assesses the
CDR outcome for the LFO
 Serves on the IPT
throughout the project

Summary
 Primary responsibility for all
oversight aspects of a MREFC
project
 Experienced or trained in
management of large projects.
 Appointed by the Division Director
(DD)
 Must not be a temporary
employee of the NSF
Conceptual Design Phase
 Determines the importance and
research priority to the affected
research community of the
science objectives motivating
consideration of a future large
facility
 Works with the research
community to develop an overall
scope for a large facility project.
 Develops the IMP
 Organizes and chairs the IPT
 Formulates a plan for eventual
divestment of the facility
 Devises and carries out a renewal
or termination strategy that
implements recompetition of the
operating award wherever
feasible

Section Revision:
May 2, 2016

2.1.6-5

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Program Officer (PO)

Grants/Agreements (G/AO) or
Contracts Officer (CO)

LFO Liaison

Preliminary Design Phase
 Works with the research
community to develop a proposal
that includes a preliminary Project
Execution Plan (PEP)
 Arranges external peer review of
the proposal
 Presents the proposed project to
the MREFC Panel
 Updates the IMP
 Continues to meet with the IPT
 Reports monthly to HLFO on
project’s technical and financial
status

 Creates solicitations for
enabling research, workshops,
summer studies, and other
activities of the research
community that will result in a
proposal (shared
responsibility with PO)
 Responsible for the business
aspects of the proposal review
and cost analysis and in
surveillance or mentoring of
the proposing institutions
 Participates in preparation of
materials for the MREFC Panel
and Director’s Review Board
(DRB)

 Advises PO
 In collaboration with PO,
plans Preliminary Design
Review (PDR)
 Independently assesses
outcome of PDR for the LFO
 Receives monthly reports on
project development from
PO, and provides
independent assessment to
the Head, LFO

 Instigates as required review,
cost analysis, or mentoring
necessary to ensure that the
recipient follows NSF business
and budgeting policies and
requirements
 Participates in periodic cost
update reviews.
 Participates in preparation of
materials for the MREFC Panel
and DRB

 Continues to monitor
project
 Receives monthly project
status updates from the PO,
adds comments and
evaluation
 Aids the PO with the
organization of the periodic
cost update reviews in
interval between PDR and
FDR.
 In collaboration with PO,
plans FDR and
independently assesses
outcome

Final Design Phase
 Continues to monitor project in
accordance with the IMP
 Provides monthly project status
updates to the HLFO
 Organizes periodic cost update
reviews
 Organizes the Final Design Review
(FDR)

Section Revision:
May 2, 2016

2.1.6-6

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Program Officer (PO)

Grants/Agreements (G/AO) or
Contracts Officer (CO)

LFO Liaison

Construction/Implementation Stage
 Develops a Cooperative
Agreement (CA)
 Approves the establishment of a
project baseline scope, cost, and
schedule and other updates to the
PEP
 Approves significant changes to
the project baseline
 Receives monthly financial and
technical status reports, quarterly
and annual progress reports
 Reports monthly to HLFO on
project’s technical and financial
status
 Conducts periodic reviews of
project progress using an external
ad hoc panel
 Arranges internal review of
Memorandums of Understanding
(MOUs)
 Regularly visits the project
 Updates the IMP
 Ensures compliance with
Government Performance and
Results Act (GPRA)

Section Revision:
May 2, 2016

 Approves submittals from
recipient
 Reviews the scope of activities
associated with each award to
ensure that the financial and
administrative framework
aligns with NSF’s expectations
for stewardship and reporting.
 Receives and provides
approval to the recipient
 Participates in baseline review
and subsequent periodic
reviews as necessary to assure
the NSF that the recipient
follows agency financial
policies
 Serves on the IPT to expedite
financial and administrative
actions and decisions
concerning the project

 Advises PO
 In collaboration with PO,
plans construction reviews
and independently assesses
outcome
 Receives monthly project
status reports from the PO
 Visits the project site
periodically in coordination
with PO

2.1.6-7

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Program Officer (PO)

Grants/Agreements (G/AO) or
Contracts Officer (CO)

LFO Liaison

Operations Stage
 Prepares and participates in
solicitation of award for
Operations and Maintenance
(O&M) CA
 Ensures compliance with GPRA
 Approves the Annual Work Plan
(which includes high level
performance goals) developed by
the recipient
 Reviews and approves the Annual
Report
 Develops budgets that operate
and maintain facilities
 Obtains Condition Assessment
reports
 Monitors planning for IT and
property security, and validates
through periodic review
 Organizes and participates in
periodic reviews of the facility
including annual operations
reviews
 Formulates a plan for eventual
divestment of the facility
 Devises and carries out a renewal
or termination strategy that
implements recompetition of the
operating award wherever
feasible

 Prepares solicitation for O&M
award (shared responsibility
with PO)
 Creates special terms and
conditions in the CA to
capture requirements for
annual performance goals
(shared responsibility with the
PO)
 Defines business practices for
renewal, recompetition, or
termination of Award
 Attends periodic reviews
including operations and
business systems reviews
(BSRs) as appropriate
 Assists in developing financial
strategy, as appropriate, to
budget for facility
maintenance and replacement
or refurbishment of long-lived
capital-assets (shared
responsibility with PO)
 Prepares Cost Proposal
Review Document (CPRD) and
performs independent cost
analyses as required

 Advises PO and G/AO on
effective operational
oversight strategies,
renewal and recompetition
strategies, and termination
 Periodically visits operating
facilities in coordination
with PO
 In collaboration with PO and
G/AO, insures
implementation of
performance measures
within the CA for operation
 Assists with organizing and
evaluating the results of
operational reviews of large
facilities
 Advises PO and G/AO on
project management issues
related to recompetition of
award for facility operation

2.1.6.2 Main Participants
Program Officer (PO)
The PO is the research community’s primary interface to the NSF. The PO’s responsibilities are
substantial, and crucial to NSF’s success. Examples of these responsibilities are listed below:1




They are typically the main contact a principal investigator (PI) has with NSF.
They are the link between what is happening in the research community and the
appropriately responsive program solicitation from NSF.
They are the catalysts for the increasing amount of research that crosses traditional
single-discipline boundaries.

1

Paraphrased from National Science Foundation: Governance and Management for the Future, a report by a panel of the
National Academy of Public Administration, April 2004. pp. 10-11.

Section Revision:
May 2, 2016

2.1.6-8

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)





They are the coaches and encouragers for proposals from less experienced researchers
– particularly ones with innovative ideas – as well as those from underrepresented
segments of the research community.
They are the recruiters and managers of a peer review process that involves numerous
experts from the research community to assess the intellectual merit and broader
impacts of proposals from the community for new research.
They are the post-award managers and monitors for awarded research.

NSF’s Authorization Act of 2002, 42 U.S.C.1862n-4I, signed into law on December 19, 2002,
restricts the choice of POs (also referred to within the NSF as Program Directors or Program
Managers) to be regular employees of the NSF. The statutory language of the Act states:
“PROJECT MANAGEMENT. No national research facility project funded under the major
research equipment and facilities construction account shall be managed by an
individual whose appointment to NSF is temporary.”
Administratively, the PO is part of a Directorate or Office that provides supervisory oversight
and the budgetary authority to fund PO actions. Depending on the administrative structure of
the originating Directorate or Office, a Section Head, Division Director, Assistant Director (AD),
or Office Head may assign a PO (or POs)1 to oversee a particular facility-related initiative and
will directly or indirectly oversee and guide the activities of the PO. Actions of the PO described
here implicitly recognize the authority of the individuals within this supervisory structure to
appropriately guide, direct, and approve the actions of the PO.
The PO exercises primary responsibility within NSF for all aspects of a large facility project,
including:







Project planning, both internally and in coordination with the relevant research
community;
Serving as the NSF interface with the research community to nurture concepts for
development and utilization by the community of a facility;
Formulating an IMP that defines NSF strategy for conducting project oversight,
managing NSF risk, and providing project funding;
Coordinating contact between the project proponents and other NSF staff members
that may need to have direct contact with the project or that the project may wish to
contact;
Chairing the IPT;

1

In some cases, more than one individual will be designated as a PO for a facility related initiative. Wherever the PO is
referenced in this manual, it should be understood that the reference is to all the relevant assigned POs.

Section Revision:
May 2, 2016

2.1.6-9

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)






Conducting merit and programmatic/technical reviews of proposals for development,
implementation, operation, and utilization of a facility (CDR, Preliminary Design Review,
Final Design Review (FDR), construction and operational reviews);
Preparing all required documentation for internal project review and approval within
the NSF;
Participating in developing the estimated costs of planning, construction, operations,
maintenance and related programmatic activities, and, under management direction of
the originating Division, Directorate, or Office, assigns budgets to these tasks; and
Overseeing implementation, operation, and eventual divestment and termination of
NSF support for the project.

Senior Management of the Originating Division, Directorate, or Office Assistant Director or
Office Head
Assistant Directors (ADs) and Office Heads lead Directorates or Offices, and by extension their
Divisions or Sections, which propose projects for funding through the MREFC Account or other
funding source.
The AD (or Office Head) of the Originating Organization utilizes community inputs, disciplinespecific studies, advisory committee recommendations and internal NSF considerations to
prioritize the opportunities represented by the candidate project relative to competing
opportunities and demands for NSF resources. The AD determines that the scientific merit and
relative importance of the proposed facility are sufficiently strong to justify advancement of the
project to Readiness Stage (i.e., ready to begin Preliminary Design activities), and authorizes the
PO to proceed with organizing the development and external review of a Project Execution Plan
and with updating the IMP to explain how NSF will oversee and fund further development. The
AD reviews and approves the IMP prior to its submission to the LFWG for review and comment.
The AD determines whether to propose a project to the MREFC Panel as a candidate for future
construction funding, based on the project’s relative scientific importance and on the
Originating Organization’s commitment to pre-construction planning activities and eventual
facility operation and use. The AD is regularly updated by the PO on the status of the project
throughout the remainder of its life cycle phases, and brings critical issues to the attention of
the NSF Office of the Director (OD) and NSB as appropriate.
The AD has overall responsibility for advancing prospective projects for consideration of
construction funding. In this capacity, the AD formulates strategic planning and budget
development within the originating Directorate or Office. This strategic planning includes
prioritizing across the research objectives of the range of disciplines served by the Directorate
or Office. The AD oversees and monitors development of NSF’s project planning, with the
assistance of supporting staff, advisory committees, and direct interactions with the broader
community affected by the facility.

Section Revision:
May 2, 2016

2.1.6-10

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

The AD oversees development of MOUs with other agencies, international partners, private
foundations, and other entities and, with the approval of the NSF OD, enters into negotiations
with those parties and signs these agreements on behalf of NSF when authority to do so is
delegated by NSF OD.
Throughout a project’s life, the AD has a primary responsibility to keep all major stakeholders in
the project informed. Interested parties include policy stakeholders (the NSF, OD); funding
stakeholders (OMB, Congress); and community stakeholders (scientific organizations and the
relevant research community).
At each stage of project development, the AD has the responsibility for making key decisions
within the originating Directorate or Office that advance a project or remove it from
consideration for further development.
Specific responsibilities include, but are not limited to:











Approving the IMP at the Directorate level;
Ensuring that the performance plans of the relevant Division Directors reflect the
requirements and expectations of the LFM and other NSF policy statements, and the
necessity to provide an environment of open communication and transparency in the
management of MREFC projects;
Assuring the evaluation and endorsement of a candidate MREFC project by the
Directorate or Office advisory committee prior to submission of the project to the
MREFC Panel for entry into the Readiness Stage;
Overseeing the organization of all design reviews including appointment of review
panels, charges to the panels, and Directorate responses to review panel
recommendations;
Reviewing and approving all Director's Review Board packages and organizing
representation of the project before NSF internal approval bodies, i.e., DRB, MREFC
Panel, and the NSB;
Representing the originating Directorate or Office in decisions to recompete
management of an operating facility, terminate support, admit new partners, and other
major decisions affecting the facility;
Assigning members of Directorate Office staff to serve as representatives on an IPT; and
Establishing appropriate Delegation of Authority for awards following NSB action.

Division Director
The Division Director (DD), assisted by Divisional Staff, has primary responsibility for overseeing
planning, review, oversight and funding of Large Facilities. This responsibility include
coordination of planning; serving as the interface with relevant scientific and engineering
communities; preparing all required documentation for project consideration and approval;

Section Revision:
May 2, 2016

2.1.6-11

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

conducting merit review of proposals; fully funding costs of operations, maintenance and
relevant programmatic activities; and overseeing the project.
Administratively, a large facility in planning, construction, or operation, is under the purview of
an Originating1 Organization, a Directorate, Division, or Office. The Originating Organization
provides supervisory oversight and budgetary authority. Depending on the administrative
structure of the Originating Organization, the cognizant PO is usually selected by the Divisional
management (e.g., Section Head and DD collaborate in the selection) with concurrence of the
AD. The PO’s superiors directly or indirectly oversee and guide the activities of the PO.
The DD has overall responsibility for the conduct of programs in a related range of disciplines
within NSF, and for the NSF interfaces between these programs and the scientific communities
in these disciplines. For large facility projects, the DD:










Evaluates and maintains, through appropriate mechanisms, the proper balance between
the totality of life cycle costs for MREFC facilities and the rest of the division’s activity;
Establishes and continually examines, through appropriate mechanisms and forums, the
priorities among MREFC candidate projects within the discipline (those in development,
under construction, and in operation);
Appoints a cognizant PO for each project;
Ensures that the program officer has the requisite experience and/or training to respond
to the responsibilities of the position;
Ensures that the cognizant PO follows appropriate best practices;
Ensures that the PO is responding appropriately to the requirements of the Large
Facilities Manual and other NSF policies and practices;
Ensures that the PO is managing interfaces with other NSF units effectively and
productively;
Ensures that the performance plan of the program officer reflects the requirements and
expectations of the LFM and other NSF policy statements; and
Facilitates the flow of information at an appropriate level of detail and timescale to keep
all NSF stakeholders appropriately informed of project progress, status, and problems.

Grants and Agreements Officer
The Grants and Agreements Officer or Contracting Officer (G/AO) has authority, subject to
statutory limitations, to award and administer CAs. The G/AO is appointed by, and is
administratively part of the DGA or DACS within the BFA. The timing of this assignment is at the
discretion of the DGA or DACS DD in response to a request from the PO, but should be early
enough in the planning stage of a large project to allow the participation of the G/AO in the
strategic planning and development of the IMP for a large project (i.e., during the Conceptual

1

This is the “lead organization” in the case where more than one Division participates in originating a project.

Section Revision:
May 2, 2016

2.1.6-12

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Design Stage when NSF begins to consider strategies for the business aspects of managing
oversight of the proposed project).
As a member of the IPT, this NSF officer participates in management reviews, risk assessment
and issues management. The G/AO plans and coordinates development of award instruments
from early planning stages through award administration and closeout. The G/AO negotiates
terms and conditions, interprets Federal and NSF policy, and reviews business proposals and
budgets, significant sub-awards, MOUs, and partnership agreements. The G/AO also monitors
awards for compliance with the most current NSF financial and administrative policies and
procedures.
The G/AO is the primary point of contact at the NSF with the recipient institution for all
business and financial matters. The G/AO represents the NSF in conducting all of the financial
and administrative business related oversight of the recipient, including:





Providing approval or authorization for all financial transactions,
Ensuring compliance with financial and administrative award terms and conditions,
Accepting submittals or reports from the recipient, and
Negotiating any specific terms and conditions which define the conduct and execution
of a project, such as CAs and subsequent amendments, MOUs, property leases, etc.

The G/AO is responsible for oversight of the financial and administrative terms and conditions
of the assistance agreement,1 just as the PO is responsible for scientific and technical oversight.
Unlike the PO, he/she holds the warrant to obligate government funds. The G/AO and the PO
jointly share the principal technical and financial responsibilities for the oversight and assurance
of a large facility project. In this capacity, the G/AO is jointly responsible with the PO for the
success of a project.
The G/AO is an integral member of the IPT for a facility project in order to expedite NSF action
on business and administrative issues related to the project.
The G/AO confers with the PO and other relevant offices to ensure that the NSF’s technical and
administrative oversight activities are well coordinated. The G/AO and the PO collaborate on
the preparation of solicitations and the proposal and award process. The G/AO has individual
responsibility for developing and overseeing the implementation of financial and administrative
aspects of the award process, and joint responsibility with the PO for recompetition planning
and execution, and award termination and closeout.

1

An assistance agreement is a grant or cooperative agreement (CA) to an institution with fiduciary responsibility for the project
or facility.

Section Revision:
May 2, 2016

2.1.6-13

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

The G/AO develops the CAs that establish a business relationship between the NSF and the
recipient. Consequently, the G/AO has an oversight responsibility that extends to the business
practices of that recipient, in addition to the specific business operations and oversight
practices of the particular project that may be based with that recipient.1
The G/AO, with the assistance of BFA resources, establishes that the financial stewardship and
reporting practices of the recipient institution, as they pertain to NSF instruments, are
consistent with NSF requirements, OMB circulars, or Federal Acquisition Rules.2
Contracting Officer
The Contracting Officer (CO) has authority, subject to statutory limitations, to award and
administer contracts for the construction and operations of facilities that are managed through
contract rather than Cooperative Agreements (CAs). The CO is appointed by the agency Senior
Procurement Executive, and is administratively part of the Division of Acquisition and
Cooperative Support within BFA. The CO is solely responsible for oversight of the terms and
conditions of the contractual agreement.
The CO holds the warrant and is the only individual authorized to obligate or de-obligate
government funds. The CO, through their warrant, has the sole authority to award and
administer the construction contract(s) used in support of Large Facility projects.
Cost Analyst
The G/AO or CO requests assistance from a NSF Cost Analyst from the Cost Analysis and Audit
Resolution (CAAR) Branch of the Division of Institution and Award Support (DIAS), located
within BFA (CAAR) when cumulative or individual awards exceed certain thresholds or for
Recipients with previously identified risks. The PO, G/AO or CO, and Cost Analyst all review
proposed budgets to help determine if they are allowable, allocable, reasonable, and realistic
for the scope of work. However, the primary purpose of the NSF Cost Analyst’s budgetary
review is to support the G/AO or CO to ensure that the Recipient has properly estimated and
calculated costs and that they are supported and documented with sufficient rigor. The Cost
Analyst provides a written recommendation to the G/AO stating whether costs are supported
or unsupported. The recommendation may include advice on award terms and conditions or
limitations or other concerns identified.
The Cost Analyst may also help determine if the Recipient has adequate business and
accounting systems in place, assess a Recipient’s financial capability and viability, validate
indirect cost rates, or assist in other areas of concern as identified by the requesting G/AO.
1 Refer to the Business Systems Review (BSR) Guide described in Section 4.5.3.3 for discussion on this point. When NSF is not the
cognizant audit agency for the recipient institution, its oversight of recipient business practices is narrowly defined.
2

Refer to the NSF -Award Monitoring and Business Assistance Program [AMBAP] and Business Systems Review [BSR] Guides for
more details on the criteria and processes for this assessment.

Section Revision:
May 2, 2016

2.1.6-14

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

While the G/AO is the primary point of contact with the Recipient for all award and cost
analysis issues, this should not inhibit direct communications between the Cost Analyst and
Recipient when necessary. Cost analysis communications with the Recipient should include the
Cost Analyst, G/AO, and PO to help ensure efficient resolution, close collaboration, and clear
and consistent direction.
Deputy Director for Large Facility Projects, (DDLFP) – [Reserved]
Head, Large Facilities Office and BFA’s Large Facilities Office
The NSF’s Head, Large Facilities Office (HLFO), and the LFO supporting staff are the NSF’s
primary resource for all policies or processes related to the development, implementation, and
oversight of MREFC projects. They are the NSF-wide resource on project management. The
HLFO has the institutional authority and resources to effectively develop mandatory policies,
which are approved by Senior Management, for all project stages. The HLFO works closely with
the BFA and NSF Senior Management Officers, providing expert opinion on non-scientific and
non-technical aspects of project planning, budgeting, implementation, and assurance to further
strengthen the oversight capabilities of NSF. The HLFO also facilitates coordination and
collaboration throughout NSF fosters the sharing of lessons learned and the use of best
practices from prior projects.
The HLFO develops and implements processes for insuring that all facility award instruments
include, at a minimum, four performance evaluation and measurement components:
1.
2.
3.
4.

Clear and agreed-upon goals and objectives;
Performance measures and, where appropriate, performance targets;
Periodic reporting; and
Evaluation and feedback to assess progress.

Prior to NSF requesting NSB approval to include a proposed project in a future budget request,
the HLFO contributes to agency assurance that the project plans are construction ready, and
that the construction and operations budgets are satisfactorily justified.1 This assurance comes
through assignment of the LFO Liaison to the IPT and membership (as assigned) on various
governance bodies such as the Director’s Review Board and MREFC Panel.
The HLFO prepares a periodic status report for NSF Leadership on all ongoing MREFC projects,
candidate MREFC projects in planning, and other large facility projects designated by the
originating Directorate or Office. Inputs to the monthly report are provided by each cognizant
PO and their associated Directorate/Division. The PO provides a monthly report that
1

See “Priority Setting for Large Facility Projects” (NSB-04-96), National Science Board White Paper, May 2004, Attachment 5 to
NSB Meeting Report, https://www.nsf.gov/nsb/meetings/2004/may_srprt.doc.

Section Revision:
May 2, 2016

2.1.6-15

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

summarizes the technical and financial status of the project, pending near-term milestones, and
any other issues that should be brought to the attention of the LFO. The HLFO combines all of
these inputs into a single report, summarizes the key technical and financial status information,
and provides an independent commentary on project management issues as necessary.
Under the direction of the NSF Senior Management, the HLFO prepares an annual National
Science Foundation Facility Plan and presents it to the National Science Board (NSB), usually at
the NSB’s February meeting. The Facility Plan describes the status and plans for the portfolio of
major multi-user facility projects that are either receiving or are candidates for receiving MREFC
funds. The Facility Plan supplements information contained in the NSF’s annual Budget Request
to Congress.
LFO Liaison
For each large facility project, the HLFO designates an LFO Liaison to work closely with the PO
and the G/AO or CO, providing expert assistance on non-scientific and non-technical aspects of
project planning, budgeting, implementation, and management to further strengthen the
oversight capabilities of NSF. The LFO Liaison participates in each project IPT and also advises
the cognizant PO of mitigating steps when project management challenges arise. The LFO
Liaison works with the PO and the G/AO, not directly with the recipient or their project staff.
The LFO Liaison also collaborates with the PO and G/AO to plan and carry out key project
reviews including CDR, PDR, FDR, Operations Reviews, and other ad hoc project reviews in all
life cycle stages as appropriate. While the PO is responsible for planning, carrying out, and
assessing the full range of topics addressed in the review, LFO Liaison focuses on project
management, business, and administrative issues, and assists the PO and G/AO in these areas.
The LFO Liaison independently assesses and reports to the HLFO on the outcome of these
reviews with respect to project management issues.
The LFO Liaison participates in site visits in coordination with the PO and originating
organization, to strengthen project management and affirm aspects of NSF’s oversight and
assurance role. During these interactions, the PO is the single point of contact with the project
for all programmatic issues, and the G/AO is the point of contact with the recipient institution
for administrative issues. Any project-specific communications between the LFO Liaison and the
project is coordinated through the respective PO and/or G/AO, and generally as part of the IPT
process.
LFO also carries out BSRs of recipient business systems for large facilities in design, construction
or operation based on a regular review cycle and other potential risks, such as building
institutional capacity in advance of an MREFC award. BSRs may also be conducted at smaller
scale facilities at the request of NSF Leadership or the originating organization. BSR objectives
and processes are described in detail in NSF’s Business Systems Review (BSR) Guide, described
in Section 4.5.3.3.

Section Revision:
May 2, 2016

2.1.6-16

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.1.6.3 Coordinating and Advisory Bodies
The Integrated Project Team
The Integrated Project Team (IPT) replaces the Project Advisory Team (PAT) and the BOT. The
PAT and BOT were advisory in nature, whereas the IPT serves as a formal coordinating body for
Large Facilities throughout the Design and Construction Stages. The IPT consists of three
primary sub-groups:
1. Science and Technical Group led by Program and primary responsible for oversight. May

include other Staff from the Division and/or Directorate as deemed appropriate by
Program (budget, science program, etc.).
2. Award Management Group comprised of various Offices and Divisions within the BFA.
This group is primary responsible for assurance. The linkage with the Science and
Technical Group is with the review and monitoring of cost, scope and schedule as well
as the Project Execution Plan. The linkage with the Strategic Group is with internal NSF
processes and procedures related to Large Facilities.
3. Strategic Group comprised of various offices within the OD. This group’s role is primarily
with assurance. The linkage with the Science and Technical Group is with
communication with external stakeholders.
The IPT is chaired by the PO. See Figure 2.1.6-4. Members are appointed by the ADs or Office
Heads, in consultation with the PO. Appointments shall be for the duration of the project or
until new appointments are made by the Office Heads. The PO will convene the IPT at least
quarterly to discuss any project-related issues.

Section Revision:
May 2, 2016

2.1.6-17

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 2.1.6-4

An Integrated Project Team (IPT), chaired by the Program Officer, is composed of three
subgroups, with appointed Award Management Group members from BFA, Science and
Technology Group members from the originating program offices, and Strategic Group
members from the Office of the Director.

Large Facilities Working Group
The Large Facilities Working Group (LFWG), previously known as the Large Facilities Panel,
serves in an advisory capacity to the LFO and is charged with:
1. Reviewing and providing comment on draft policies, processes and procedures related
to NSF Large Facilities including, but not limited to, the LFM
2. Raising, discussing and providing inputs on issues related to NSF Large Facilities
3. Review and comment on IMPS prior to the CDR
The Working Group is chaired by the Head, Large Facilities Office (HLFO). Members are
appointed by the ADs, in consultation with the DDLPF, to ensure that the overall make-up
provides a diverse range of perspectives from programmatic and project management to
Directorate-level interests. Appointments shall be for a term of one (1) year.
A member of the LFO staff will serve as the Group’s executive secretary. Ex-officio members
include representatives from the OD, the Cooperative Support Branch (CSB) of the Division of
Acquisition and Cooperative Support (DACS), and the Office of the General Counsel (OGC).

Section Revision:
May 2, 2016

2.1.6-18

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

The HLFO will convene the Group at least quarterly to discuss issues raised by the Group or the
agency, and set priorities related to review of documentation related to Large Facilities.
Advisory Committee of the Originating Directorate or Office
The Advisory Committee of the Originating Organization provides input to the NSF AD, or Office
Head of the Originating Organization concerning priorities among and between projects and
other activities sponsored by the Directorate. The NSF Director requires the endorsement of
the Advisory Committee of the Originating Organization prior to requesting NSB action
approving a project’s inclusion (at the Director’s discretion) in a future NSF budget request to
Congress.
2.1.6.4 Governing Bodies
MREFC Panel
The MREFC Panel ensures that the overall MREFC process is followed. It reviews specific cases
as presented by the Originating Organization(s) and makes recommendations to the Director.
The Panel consists of the NSF Deputy Director (Chair), the Ads, Program Office Heads, the Chief
Financial Officer, the other Senior Management of NSF, and (in non-voting capacity) the HLFO.
The MREFC Panel assesses and prioritizes major research infrastructure projects funded
through the MREFC account.1 The Panel reviews specific on-going and candidate projects as
presented by the Originating Organization and makes recommendations to the NSF Director. In
particular, because the Panel is composed of administrators with responsibility for every area of
science and engineering supported by NSF, it applies the third ranking criteria2 for prioritizing
MREFC projects, which pertain to overall NSF priorities:







Which projects are in new and emerging fields that have the most potential to be
transformative? Which projects have the most potential to change how research is
conducted or to expand fundamental science and engineering frontiers?
Which projects have the greatest potential for maintaining U.S. leadership in key science
and engineering fields?
Which projects produce the greatest benefits in numbers of researchers, educators and
students enabled?
Which projects most need to be undertaken in the near term? Which ones have the
most current windows of opportunity, pressing needs and international or interagency
commitments that should be met?
Which projects have the greatest degree of community support?

1

For example, the MREFC Panel reviews the Large Facilities Manual and supporting information, such as this document. The
Large Facilities Manual and its supporting materials are “living materials” that are periodically updated to reflect additional
requirements and/or policy changes as they are reviewed by the MREFC Panel, NSF Director and the National Science Board.
2

See Appendix A of the Large Facilities Manual – Ranking Criteria for Prioritizing MREFC Projects.

Section Revision:
May 2, 2016

2.1.6-19

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)



Which projects will have the greatest impact on scientific advances across fields taking
into account the importance of balance among fields for NSF’s portfolio management in
the nation’s interest?

Under the guidance of the NSF Director and Deputy Director, the MREFC Panel reviews the
current status, planning and implementation, challenges and concerns, and any policy issues
concerning MREFC projects throughout the year. The MREFC Panel recommends advancement
of projects into successive stages of development, and may provide other review and
assessment as directed by the Deputy Director.
Director’s Review Board
The purpose of the Director’s Review Board (DRB) is to assure the Director that all
recommendations and proposed action items have undergone thorough review, assessment
and discussion. The DRB reviews proposed actions for adequacy of review and documentation
and for consonance with Foundation policies, procedures and strategies. The DRB also brings to
the Director’s attention any policy issues that have been identified.
The DRB is the Director’s forum for reviewing timely recommendations to the NSB on a variety
of critical NSF awards, actions, and information items, including those related to large facilities.
The DRB reviews for responsiveness to questions that may be raised by the NSB.
Members of the DRB may include:
•
•
•
•
•
•

Chairperson (NSF Deputy Director or other);
Three ADs, serving on a rotating basis;
Chief Financial Officer;
Staff Advisor, OD;
Executive Secretary, DRB; and
Such other persons as the Director may designate (i.e., OGCs, Legislative and Public
Affairs, etc.).

Joint meetings between the DRB and MREFC Panel may be scheduled as the particular situation
warrants, but keeping in mind their distinct roles and responsibilities.
NSF Director
The NSF Director has ultimate responsibility for the approval of the obligation of funds from the
MREFC Account and for proposing new MREFC projects to the NSB, OMB and Congress. The
Director approves all materials submitted to the NSB, OMB or Congress.

Section Revision:
May 2, 2016

2.1.6-20

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

National Science Board
The National Science Board (NSB) establishes policy, reviews and approves MREFC Account
budgets, and reviews and approves specific large awards for funding, including MREFC
projects.1 NSB is an independent body established by Congress in 1950 to set policies for NSF.
Within NSB, the Committee on Programs and Plans (CPP) oversees NSF program initiatives and
major new projects and facilities. The NSB sets the priority order of projects recommended for
construction.
The NSB oversees NSF and establishes NSF policies within the framework of applicable national
policies set forth by the President and the Congress. In this capacity, the NSB identifies issues
that are critical to NSF’s future, approves NSF’s strategic directions, annual budget requests,
new major programs and awards, and provides guidance on the balance between initiatives,
infrastructure investments and core programs.2
The NSB has established a process for reviewing and approving recommended actions and
funding requests from NSF regarding large facility projects during facility development.3 The
NSB performs certain reviews and approvals, including an annual review of facilities, and
prioritizes projects as necessary. NSB involvement at each life cycle stage includes:






Sets policies of NSF that determine the administrative framework for overseeing all life
cycle stages s of NSF’s large facilities;
Is kept apprised of the status of all large facilities funded by NSF through oral and
written information items, particularly projects in the development and construction
stages. Approves awards for advancement through design phases if above the NSB
threshold;
Approves the release of NSF’s annual Facility Plan;
May provide guidance or expectations for pursuing further development of a project,
which, if not realized, could result in terminating further NSF support; and concurs in
any recommendation to terminate support;

1

The Proposal and Award Manual (PAM) requires the following items to be submitted to the NSB for approval: (1) Large
Awards. Proposed awards where the average annual award amount is 1% or more of the awarding Directorate or Office's prior
year current plan (including any funds transferred from other Federal agencies to be awarded through NSF funding actions);
(2) Major Construction Projects. NSB approval is required when the resulting cost is expected to exceed the percentage
threshold for NSB award approval; (3) Awards Involving Policy Issues or Unusual Sensitivity. NSB interests may include the
establishment of new centers, institutes, or facilities that have the potential for rapid growth in funding or special budgetary
initiatives.
2

More about the NSB is available online at https://www.nsf.gov/nsb/

3

See the internal NSF document “NSB MREFC Process” (graphic, NSB-/CPP – 12-18, approved May 4, 2012). See also NSB’s
meeting minutes with “Annual Timeline for Integration of Board MREFC Process with NSF Budget Process” (NSB-10-66,
approved August 2010).

Section Revision:
May 2, 2016

2.1.6-21

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.6 Roles and Responsibilities for NSF Staff for Management and Oversight of Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)








May recommend augmentation of the budget of the Division originating a candidate
new facility, to partially offset the impact on other programs resulting from the need to
fully support pre-construction planning;
Recommends inclusion of a candidate project in a future NSF Budget Request to
Congress, after a PDR and NSF Director approval;
Prioritizes the order of construction start among projects similarly approved for
inclusion and not yet started;
Authorizes the Director to obligate appropriated MREFC construction funding to the
recipient;
Reviews all recommendations for awarding funds to operate large facilities if above the
NSB approval threshold; and
Approves recompetition strategies for operations awards if above the NSB approval
threshold.

Section Revision:
May 2, 2016

2.1.6-2

Large Facilities Manual: NSF 16-0XX (May 2016)
2.2.1 Initiation of a Potential MREFC Project
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.2

DEVELOPMENT STAGE

2.2.1 Initiation of a Potential MREFC Project
As in all NSF endeavors, inquiry begins with the research communities, whose members alert
NSF program staff to the most promising and exciting questions and the most important
equipment needed to explore them.
NSF POs, who work closely with those communities, should be attentive to the emergence of
breakthrough concepts and actively encourage discussion and planning. In addition, NSF uses
National Academies’ studies, community workshop reports, professional society activities,
Directorate advisory committees and many other methods to identify opportunities and ensure
continuous community input.
Ideas and opportunities identified by the research communities typically have a 5- to 20-year
forward look and are brought to NSF in the form of a submitted proposal requesting funding for
development. When there are competing concepts, it may be appropriate for NSF to issue a
solicitation inviting proposals.
In most cases, program staff will take a proactive role in facilitating proposal submission, merit
review, recommendations and decision. In so doing, however, a PO should maintain the
position of a neutral, unbiased agent of NSF. Project advocacy should rightly come from the
community, which also participates in the merit review process and whose input is a significant
contributing factor in NSF’s funding decisions.
During the early development stage, there should be sufficient investment by the Originating
Organization (Directorate and/or Division) so that the project is reasonably well defined and/or
described in preparation for the formal design stage.

Section Revision:
June 17, 2015

2.2.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.2.2 Exit from Development to Design Stage
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.2.2 Exit from Development to Design Stage
Formal start of the Design Stage for a facility project occurs once a recommendation of the
MREFC Panel and approval by the NSF Director is received. This process is initiated by a request
from the sponsoring Directorate and/or Division to the Director’s Office once a project is
determined to be ready for the Conceptual Design Phase and potential construction with
MREFC funds. Generally, such a request is made when the sponsoring organization has
determined that: (1) the project is a high priority for further development, (2) the project is
eligible for MREFC funding (see criteria) and the MREFC funding route is preferred, and (3) the
sponsoring organization is committed to begin explicit investment in more detailed design
activities in the current or upcoming budget cycle using Directorate or Divisional funding
(R&RA).
The MREFC Panel’s recommendation will focus on providing the Director with answers to the
following questions:
Science



Is there a compelling science case, and are the project’s goals well-articulated?
Does the project fit solidly within the NSF “mission,” within the strategic plans of the
NSF and that of the sponsoring Directorate or Division, and within the broader NSF
facility portfolio?

Planning





Is the sponsor’s plan for stewardship of the Conceptual Design Phase consistent with the
guidelines set out in the Large Facilities Manual?
Does the preliminary timeline for development and implementation include
programmatic, NSB, budget and any necessary partnering milestones, including explicit
project off ramps?
Are potential opportunities for internal and or external partnering being considered, if
not already underway?
Are there any conflicts of interest or other major challenges regarding this project that
the Director needs to be aware of?

Based on the Panel’s recommendation and any further examination, the Director then approves
(or disapproves) the project entering the Conceptual Design Phase as a “candidate” MREFC
project. Note that no NSF commitment is implied beyond support for the development of a
Conceptual Design. The MREFC Panel or Director might alternatively advise the sponsoring
organization to look further into an issue or issues and then return to the Panel for further
consideration.

Section Revision:
June 17, 2015

2.2.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.1 Conceptual Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.3

DESIGN STAGE – CONCEPTUAL, PRELIMINARY, AND FINAL DESIGN PHASES

2.3.1 Conceptual Design Phase
2.3.1.1 Introduction – Conceptual Design Phase
The goal of this first phase of the MREFC design stage is the creation of a comprehensive
Conceptual Design that clearly articulates project elements that NSF will consider, including:









Description of the research infrastructure and technical requirements needed to meet
the science, including a definition and relative prioritization of the research objectives
and science questions the proposed facility will address. Technical requirements must
flow down from the science requirements. This description may be site-independent or
site-specific depending on the nature of the project;
System-level design, including definition of all functional requirements and major
systems;
Initial risk analysis and mitigation strategy for construction, identifying enabling
technologies, high-risk or long-lead items, and research and development (R&D) needed
to reduce project risk to acceptable levels;
Potential environmental and safety impacts to be considered in site selection (see
“Compliance with Environmental, Cultural and Historical Statues,” at the end of this
section);
Description of the proposed performance baseline (scope of work, budget and schedule)
needed to evaluate readiness and continue planning in preparation for the Preliminary
Design Phase. This includes budget and contingency estimates appropriate to a
Conceptual Design1 and based on the initial Risk Analysis and initial projections for the
construction and commissioning schedule;
Description of proposed Educational Outreach and Broader Societal Impact, included in
the proposed scope of work, budget and schedule.

Many of these details are included as part of the PEP as described in greater detail in following
sections and in Section 3.4. This Phase may take several years depending on development
activities.
2.3.1.2 Conceptual Design Phase Activities
During the Conceptual Design phase there may be a number of coordinated and complimentary
activities taking place with the various entities involved: (1) community activities, (2) NSF staff
activities, and (3) funding considerations.

1

The budget information should be provided using a Work Breakdown Structure (WBS) format, identifying the basis for
estimates and including a WBS dictionary that defines the scope associated with each WBS element. Contingency estimates
should include an explanation of the methodology used to calculate the estimate.

Section Revision:
May 2, 2016

2.3.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.1 Conceptual Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

(1) Community Activities. Proponents of a project should provide NSF with an early concept
proposal that makes a compelling case for the research that would necessitate development of
a facility, and that describes, in general terms, its essential characteristics if the proposal is
unsolicited. Generally speaking, large facilities projects are solicited. In that case, the proposal
must respond to all NSF and programmatic requirements which generally include references to
the Large Facilities Manual if it is already known as a candidate MREFC project. These initial
proposals identify what is known at that point in project development, as well as what tasks
remain to be accomplished in order for NSF to consider a project for eventual funding. In the
near term, they also define what work should be done to develop the project to the Conceptual
Design level of maturity.
An NSF PO1 will be assigned to be the primary point of contact with the Principle Investigator
(PI) and/or Project Manager. The NSF PO conducts a merit and technical/programmatic review
of the proponents’ proposal, and either recommends or declines the request for funding. If
funded, the PO will work with their Directorate and/or Division to organize an Integrated
Project Team to provide coordination on project oversight and assurance.
Proponents should acquaint themselves with NSF’s expectations for the essential elements of a
construction-ready PEP as described in Section 3. Proponents should also develop a skeletal
plan that will result in the future definition of each of these elements, should NSF encourage
further pre-construction planning. The plan should address, even if only in the most cursory
way, each of the essential elements that should be realized in a formal construction-ready PEP.
For example, proponents may wish to develop a “straw man” PEP that contains sections labeled
using each of the entries in Section 3.5, with as much supporting information provided based
on the outputs from the Development Stage (if any) and/or the requirements in the solicitation.
This serves to illustrate an understanding to all parties of the range and magnitude of the tasks
ahead.
(2) NSF Staff Activities: In response to the development of an early version of a PEP, the PO,
with the advice of the IPT, develops an IMP.2
This internal document specifies how NSF will conduct its oversight and assurance of the
project, and provides budgetary estimates for developing, constructing and operating the
1

Administratively, the Program Officer (PO) is part of a Directorate or Office that provides supervisory oversight and the
budgetary authority to fund PO actions. Actions of the PO described here and in subsequent life cycle stages of facility
development implicitly recognize the authority of the individuals within this supervisory structure to appropriately guide, direct,
and approve the actions of the PO. In particular, when the phrase “PO concurrence” is used in the following text, this assumes
concurrence at whatever management level the AD or Office Head has required. Refer to Section 2.1.6 for a brief description of
the duties of the PO, AD, and others referred to in the Large Facilities Manual.
2

See the NSF cognizant PO for a discussion of the internal document “Guidelines for Development of Internal Management
Plans for Large Facilities.”

Section Revision:
May 2, 2016

2.3.1-2

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.1 Conceptual Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

facility. It also identifies critical issues and risks facing the project (for example: project
management issues, completing essential R&D activities, partnership agreements, termination
or divestment liabilities) and lays out a strategy for financing these activities.
The PO develops the IMP with advice and assistance from the IPT. Following consultation,
review and approval within the sponsoring NSF Division and/or Directorate, and upon approval
of the IMP by the cognizant NSF AD, the IMP is formally reviewed by the LFWG.1 The LFWG is
chaired by the HLFO and includes other NSF staff members experienced in the technical and
administrative aspects of large project oversight. The LFWG provides written comments on the
IMP, which become part of the review record and are available to the PO, the Originating
Organization, the MREFC Panel and the Director.
The IMP describes the plan for NSF management and funding of the project to CDR, proposes
transitional steps to be taken if the project is admitted to the Preliminary Design Phase, and
lays out NSF’s plan to oversee development of the project including internal and external
review. Each large project undertaken by NSF has unique characteristics. Accordingly, the IMP
should be adapted to meet the specific needs of a particular project. The IMP should state the
justification for pursuing alternatives to the guidelines contained in the Large Facilities Manual.
3) Funding Considerations. During the Conceptual Design Phase, NSF and/or other institutions
and agencies begin to invest research and development funds in design development, and in
efforts that promote community building and planning. Investment in fundamental research
activities, community building, and initial planning activities may occur over many years, and
some are recognized as having contributed to the conceptual design effort only in retrospect.2
The cumulative pre-construction investment in research, planning and development that occurs
during the Conceptual Design, Preliminary Design, and Final Design phases may range from five
to 25 percent of total construction cost, depending on the complexity of the project, and
typically amounts to about 10 percent of the construction cost. The technology needed to
construct a facility may be uncertain, unproven or immature, requiring substantial development
over a period of years.
NSF may decide to fund additional planning and development efforts for particular projects
depending on the outcomes of the review and whether or not the Conceptual Design Phase was
funded.3 Such activities might include workshops in one or more disciplines, National

1

The composition of the LFWG is given in Section 2.1.6.

2

Some projects come to NSF very well developed, requiring little in the way of conceptual design phase support. They are
subjected to the same rigorous scrutiny, however, as they are developed by the responsible NSF Directorates or Offices.
3

Relevant program solicitations may be released to announce funding opportunities for these planning and development
efforts.

Section Revision:
May 2, 2016

2.3.1-3

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.1 Conceptual Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Academies’ studies, and research projects related to the development of new technologies.1
These activities might be funded as part of the Conceptual Design Phase award, or through a
separate proposal submission.
2.3.1.3 Conceptual Design Review (CDR)
The Conceptual Design Phase is complete when a package containing the Conceptual Design
and funding request leading to a Preliminary Design is received, reviewed, negotiated and
approved for funding. The funding request will generally be submitted as a supplemental
request to the original award.
The package should include the refined PEP and any additional information required by
Program to assess the project readiness and management to-date. Components of the PEP are
given in Section 3.4.
NSF will subject the Conceptual Design package to external review, applying standard NSF
criteria (Intellectual Merit and Broader Impacts) as well as other programmatic and technical
criteria as given in the original solicitation and the panel charge. Projects that review well will
be further evaluated by NSF to apply the second ranking criteria (agency strategic fit), in
accordance with the principles stated in the joint NSB/NSF Management Report: Setting
Priorities for Large Research Projects Supported by the National Science Foundation
(NSB-05-77). (See Appendix A for discussion of ranking criteria.)
The review panel will, as appropriate, involve external experts, consulting firms, and in-house
expertise in the science, technology and business communities to scrutinize and validate the
supporting planning documents. The scope of this review includes assessment of the scientific,
technical and project-management aspects of the proposal.
The review is organized and conducted by the PO in consultation with the LFO Liaison. The PO
has overall responsibility for organizing the review, and throughout the review process acts as
the interface between the NSF and the recipient. The PO authors the review charge and
organizes the review panel. The LFO Liaison strengthens the review process by specifying
language for incorporation within the charge and for aspects of the review agenda pertaining to
project management issues, and recommending panelists able to advise NSF in non-science
related areas of the review. The PO and the LFO Liaison concur on the implementation of these
recommendations. Following the review, the PO and the LFO Liaison will each independently
assess the review, confer on areas of concern, share their views, and report their observations
through their respective supervisory chains – the PO via the administrative structure of the
sponsoring Directorate or Office and the LFO Liaison via the HLFO.
1

NSF encourages disciplinary and interdisciplinary science planning by all of the research communities that NSF supports. In
particular, NSF encourages formal planning in fields in which scientists and engineers have traditionally not been organized to
identify MREFC projects needed for breakthrough advances.

Section Revision:
May 2, 2016

2.3.1-4

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.1 Conceptual Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

At this point, the conceptual design baseline is likely to have significant uncertainties. Cost
estimates at CDR are generally parametric in nature. Contingency estimates, representing work
scope not yet defined but nevertheless essential to the completion of the project, will be a
significant fraction of the total project budget estimate. Significant unknowns and uncertainties
often remain to be addressed in more advanced stages of planning and development. The
conceptual design, system requirements, supporting budget estimates, risk analysis, and
forecasts of interagency and international partnerships should be detailed enough for NSF
program officials to decide whether the project concept warrants further funding for
development.
Immediately following CDR, the initial high-level NSF Cost Analysis will be initiated and
conducted jointly with key assurance members of the IPT; namely DACS, the Division of
Institution and Award Support (DIAS), and the LFO. The Cost Analysis will be conducted following
NSF internal Standard Operating Guidance (SOG). Guidance on refinements to the recipient’s
Cost Book will be provided as necessary in preparation for the Preliminary Design Phase.
2.3.1.4 Exit from the Conceptual Design Phase
Formal exit from the Conceptual Design Phase typically entails three NSF actions:
1. Successful completion of the CDR as described above,
2. Recommendation for advancement by the sponsoring Directorate, and
3. Approval for advancement to the Preliminary Design Phase by the OD
Recommendation for Advancement by the Sponsoring Directorate
The AD relies on community inputs, discipline-specific studies, advisory committee recommendations and internal NSF considerations to prioritize the opportunities represented by the project
relative to competing opportunities and demands for resources. If, in the judgment of the AD, the
scientific merit and relative importance of the proposed facility are sufficiently strong to justify
advancement of the project into the Preliminary Design Phase, the AD will submit a
memorandum to the MREFC Panel recommending the project for support, that explains how it
meets the requirements for MREFC funding and how it satisfies the following criteria:



The project’s science (research) program addresses one or more science objectives in
the current NSF Facility Plan, clearly demonstrating a compelling need for the project;
The project has been reviewed by the research community and by NSF, in consultation
with Directorate Advisory Committees, and has been assigned a very high priority;1 and

1

Evaluation by NSF includes external merit review, using the NSF merit review criteria and the 1st ranking Criteria in Appendix
A and evaluation by the MREFC Panel, using the 2nd ranking Criteria.

Section Revision:
May 2, 2016

2.3.1-5

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.1 Conceptual Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)



The project’s CDR indicates that: (1) the engineering design and construction plans are
appropriately defined at the conceptual design level of project maturity and that the
management plans and budget estimates for further planning and development, as well
as constructing and operating the facility are reasonable; (2) the sponsoring Directorate
endorses the IMP and Project Development Plan1 (PDP) for further development to the
Preliminary Design/Readiness Phase; (3) the technology to create the facility exists or
can exist shortly, and can be used without excessive risk; (4) other risks to development
are satisfactorily defined and minimized or otherwise addressed in the IMP, and (5)
there are no better alternatives to the facility (i.e., with a better mix of cost and quality)
that would address the science objectives in a timely manner.

Supporting documentation, including the approved IMP, relevant review evaluations, and any
other supporting information should accompany this memorandum. All materials are
transmitted to the MREFC Panel by the AD or Office Head of the sponsoring Directorate or
Office. On the basis of this documentation, a presentation by and discussions with NSF program
staff, the MREFC Panel reviews candidate projects, assessing the relative merit of the candidate
scientific or engineering research facility in comparison to other projects and opportunities
competing for NSF resources, and recommends to the Director those projects that should move
into the Preliminary Design/Readiness Phase.2
Approval by the Office of the Director (OD)
The Director evaluates the MREFC Panel recommendation and, if satisfied, approves
advancement to the Preliminary Design Phase. The project is then included in the Facility Plan,
which is released annually.
At its May meeting, the NSB’s Committee on Strategy and Budget reviews the portfolio of
projects which are being considered for future funding and evaluates relative priorities that
guide NSF’s investment looking across the entire range of disciplines served by NSF within the
constellation of other competing opportunities, existing facilities, and the balance of support
for infrastructure, its utilization, and individual investigator-led research. The NSB is asked to
concur with the Director’s decisions by approving the annual Facility Plan.
More information about the role of the NSB in selecting and prioritizing large facility projects is
available in Section 2.1.6 on Roles and Responsibilities.

1

The Project Development Plan is part of the PEP, providing the plan to develop the project design and definition to readiness
for construction. See Section 3.4 for details.
2

When an Originating Organization(s) proposes more than one candidate project for consideration by the Panel within a twoyear time frame, it should prioritize its slate of projects and provide a rationale for its recommendations to the Director.

Section Revision:
May 2, 2016

2.3.1-6

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.2 Preliminary Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.3.2 Preliminary Design Phase
2.3.2.1 Introduction – Preliminary Design Phase
The Preliminary Design/Readiness Phase further develops concepts to a level of maturity in
which there are: a fully elaborated definition of the motivating research questions; a clearly
defined site-specific scope; a PEP and an IMP that address major anticipated risks in the
completion of design and development activities
and in the undertaking of construction; and an
“Off-ramps”
accurate budget estimate that can be presented
Projects may be removed from the Preliminary
Design/Readiness Phase by the NSF Director
with high confidence to the NSF Director, NSB, the
due to:
Office of Management and Budget (OMB), and
Congress for consideration for inclusion in a future
 Insufficient priority over the long term;
NSF budget request. Outcomes from the
 Failure to satisfy milestones or other
Preliminary Design Review are what establish the
criteria defined in the imp/pep;
 Eclipse by other projects;
project baseline.
NSF has implemented a “no cost overrun policy”
on MREFC-funded construction projects. This
policy requires that the Total Project Cost (TPC)
estimate developed at the Preliminary Design
Stage has adequate contingency to cover all
foreseeable risks, and that any cost increases not
covered by contingency be accommodated by
reductions in scope.1









Collapse of major external agreements;
Extensive estimated or actual cost
overruns;
Significant changes in schedule for
development;
Unexpected technical challenges;
Changes in the research community that
indicate eroding support for the project;
or
Any other reason that the director deems
sufficiently well-founded.

To satisfy these requirements, the project is
Specific reasons for removing an MREFC
2
project from this phase will be made public via
developed to a Preliminary Design level of
maturity. Results of this development are reflected the NSF Facility Plan.
in a revised and updated PEP.3 Components of the
updated PEP that deserve particular emphasis at this stage include:



1

Update of the project development plan budget and timeline, with major anticipated
risks in the completion of design and development activities;
Refinement of the research objectives and priorities of the proposed facility;

See the MREFC Section of the NSF’s 2009 Budget Request to Congress, page 3, available online.

2

NSF utilizes the conventional definition of preliminary design as used by project managers – a site-specific design defining all
major subsystems and their interconnections, a level of design completeness that allows final construction drawings to
proceed, cost estimation based on construction bidding, and bottom-up estimates of cost and contingency. Preliminary design
usually has a specific meaning within a particular industry or discipline, and NSF adopts the definition most appropriate to each
particular project, as defined in the Project Development Plan part of the PEP.
3

See Section 3.4 for a description of the PEP.

Section Revision:
May 2, 2016

2.3.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.2 Preliminary Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)














Update of the description of the required infrastructure, site-specific design, and
definition of interconnections of all major subsystems;
Environmental Assessments or Environmental Impact Statement (if applicable);
Bottom-up budget and contingency estimates for construction, presented using a Work
Breakdown Structure (WBS) structure and supported by a WBS dictionary defining the
scope of individual elements;
Updated construction schedule with contingency estimate;
Updated Educational Outreach and Broader Societal Impact plan that includes the scope
of work, required budget and schedule to implement the plan, plus the budget and
schedule needed to develop the plan from preliminary design to final design;
Implementation of a Project Management Control System (PMCS) 1 and inclusion within
the preliminary design of a resource-loaded schedule;
Updated risk analysis, including regulatory issues affecting construction or operation,
and time-dependent factors such as inflation indices, price volatility of commodities,
etc. (The preliminary design budget estimate will be the basis for a future NSF budget
request to Congress if the project successfully emerges from the Preliminary
Design/Readiness phase. Costs and risks should be projected forward to the anticipated
award date for construction funds.)
Demonstration that key technologies are feasible and can be industrialized if required;
Plans for management of the project during construction, including preliminary
partnership arrangements and international participation, oversight of major subawards and subcontracts, organizational structure and management of change control;2
and
Updated estimates for future operating costs, anticipated future upgrades, or possible
decommissioning costs of the facility at the end of its operating life.

2.3.2.2 Preliminary Design Phase Activities
During the Preliminary Design Phase, the earlier conceptual design evolves into a more mature
plan with respect to the baseline and contingency definitions. The WBS elements and resource
estimates benefit from additional knowledge and planning. Consequently, budget uncertainty
for projected construction is much reduced relative to the earlier conceptual design. At the end
of the Preliminary Design Phase, the approved total project cost (performance baseline
estimate plus contingency) is capped. (Additional planning and development during the final
pre-construction design stage may result in exchanges between contingency budget and
1

The PMCS involves both the software tools for development of the project databases and the processes and procedures
needed to organize and manage the project; schedule and optimize project resources;; compute and track Earned Value and
evaluate project risk factors; and manage the change process by evaluating the effects of alterations to the baseline on the
project’s planned budget and schedule.
2

These plans are a preliminary, but relatively mature version of the Project Execution Plan that defines how the project will
conduct itself during the construction stage – see Section 3.4.

Section Revision:
May 2, 2016

2.3.2-2

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.2 Preliminary Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

performance baseline, but the total project cost does not change.) Typically, a significant
proportion (often one-third or more) of the total pre-construction planning budget is expended
achieving the preliminary baseline.
Interim reviews1 during the Preliminary Design Phase will be conducted by NSF as described in
the IMP. This stage culminates in a Preliminary Design Review (PDR), conducted by NSF, to
ensure that all aspects of the project definition and planning are robust. The results of the PDR
are reported by the MREFC Panel, followed by a recommendation to the Director for decision
on forwarding to the NSB for possible inclusion in a future MREFC budget request.
2.3.2.3 Preliminary Design Review (PDR)
NSF conducts a PDR, organized and led by the PO, to assess the robustness of the technical
design and completeness of the budget and construction planning. Like CDR, the review is
organized and conducted by the PO in consultation with the LFO Liaison. The PO has overall
responsibility for organizing the review, and throughout the review process acts as the interface
between the NSF and the recipient. The PO authors the review charge and organizes the review
panel. The LFO Liaison strengthens the review process by specifying language for incorporation
within the charge and for aspects of the review agenda pertaining to project management
issues, and recommending panelists able to advise NSF in non-science related areas of the
review. Following the review, the PO and the LFO Liaison will each independently assess the
review, confer on areas of concern, share their views, and report their observations through
their respective supervisory chains – the PO via the administrative structure of the sponsoring
Directorate and the LFO Liaison via the HLFO.
The review scrutinizes the effectiveness of project management through this phase of
development, as well as plans for completion of final design and eventual construction and
operation. The PDR may utilize, as appropriate, external experts, consultants and outside firms
to evaluate proposed plans and budgets. The PDR also examines the management structure
and credentials of key staff to assure NSF that an appropriately skilled management
organization is ready to complete final design activities and execute the construction phase of
the project.
Once the project has satisfied any recommendations made by NSF as a result of external
review, and resolved any outstanding issues, the Directorate recommends to the MREFC Panel
that the project is ready for advancement to the Final Design Phase of development and is a
candidate for NSB approval for inclusion in a future NSF budget request for construction
funding. At any time, the MREFC Panel or the OD may request further external review.

1

Interim reviews are typically held semi-annually. Exceptions to this, dictated by the needs of a particular project, are justified
in the IMP.

Section Revision:
May 2, 2016

2.3.2-3

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.2 Preliminary Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Following the PDR, the PO updates the IMP to describe proposed plans for budgeting and
oversight, and to finalize commitments from interagency and international partners during final
design. The PO directs the recipient to update the PEP to lay out the work scope, budget and
schedule necessary to bring the project to Final Design.
Immediately following the PDR, the second, more detailed NSF Cost Analysis will be initiated
and conducted jointly with key assurance members of the IPT; namely the Division of
Acquisition and Cooperative Support (DACS), the DIAS, and the LFO. The Cost Analysis will be
conducted following NSF internal SOG. Guidance to the Recipient on refinements to the Cost
Book will be provided as necessary in preparation for the Final Design Phase.
The completion of project planning and development, culminating in a Final Design, should be
aligned with the expected time-scale for requesting and appropriating construction funds. The
NSF Budget Office is the coordinator for this critical planning activity, bringing projects forward
for construction only if OMB and Congress are likely to approve the request and appropriation
of funds within the time period in which the Preliminary Design plans and cost estimate remain
valid.
2.3.2.4 Exit from Preliminary Design Phase
A candidate project exits from the Preliminary Design phase and enters the Final Design phase
after the following have been completed:
1. A successful PDR and subsequent support from the Directorate,
2. A review and recommendation by the MREFC Panel for advancement to the Final
Design Phase,
3. The NSF Director approves advancement and recommends to the NSB inclusion of the
project in a future year budget request, and
4. The NSB approves inclusion in a future MREFC budget request.
2.3.2.5 NSF Director’s Recommendation for Advancement to Final Design
The MREFC Panel and the Director should first be satisfied that the following conditions have
been met before making a recommendation to the NSB for approval:




The AD of the sponsoring Directorate continues to assert the high scientific merit and
importance of the project and has a sound financial plan for supporting the remaining
pre-construction planning activities and the future operations and use of the facility.
The Preliminary Design has been successfully reviewed internally and by an external
panel of experts in order to obtain the best possible objective advice from authorities in
the fields and disciplines utilized by the project.

Section Revision:
May 2, 2016

2.3.2-4

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.2 Preliminary Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)







The MREFC Panel concurs that the Preliminary Design is reasonable and poses an
acceptable level of risk, and that anticipated costs for construction and operation are
sufficiently well known.
An appropriate Integrated Project Team (IPT) is in place and has provided assurance that
the Preliminary Design Total Project Cost has been satisfactorily analyzed at a high
degree of confidence to support the budget request.
The NSF Director is satisfied that external participation in all phases of the project (other
agencies, international and/or private sector entities, etc.) is well planned.
Updated IMP and PEP have been reviewed and approved by the Large Facilities Working
Group (IMP only) and the IPT.
The MREFC Panel asserts that the proposed MREFC project, when compared to other
proposed projects – whether within the same field, across related fields, or across
different fields1 – is among the very highest priorities for potential new facilities.

Based on its review of the information provided and discussions with Program and the
sponsoring Directorate, the MREFC Panel forwards one or more projects in priority order to the
Director, who makes the decision on advancement and whether or not to forward to the NSB
for approval. The rationale and criteria used for the selection and prioritization of these
projects is clearly articulated in the Facility Plan.
2.3.2.6 National Science Board Approval
The final steps for exit from the Preliminary Design Phase are review and approval by the NSB
for advancement into the Final Design phase and inclusion in a request to the OMB for future
year funding.
The Originating Organization is responsible for preparing the documentation needed for the
NSB to review and approve a proposed MREFC project for advancement to Final Design and
inclusion in a future budget request. Prior to NSB submission, the Director’s Review Board
(DRB)2 reviews the completeness and appropriateness of the documentation supporting
advancement of the project (such as prior phase reviews, committee evaluations, PEP
evaluation and reviewed proposal ratings) to ensure adherence to NSF processes and policies.
As NSB considers projects for advancement to Final Design, NSF makes available to the NSB,
upon request, the PEP and IMP, and the reviews from the community, the Large Facilities
Working Group, the LFO, the MREFC Panel and other relevant parties. The NSB considers the
following elements, applying primarily the third ranking criteria (national priorities: see
Appendix B), as appropriate:

1

In making this determination, the second and third ranking criteria in Appendix A are judiciously applied.

2

See Section 2 and Section 2.1.6 on Roles and Responsibilities.

Section Revision:
May 2, 2016

2.3.2-5

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.2 Preliminary Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)







The research and science enabled by the proposed facility;
Construction plans together with their risks and degree of readiness;
Budget justification for construction and operation of the facility;
The likelihood that funding will be available in the next few years; and
The priority of the project in furthering one or several objectives in the Facility Plan.

As with all NSF proposals, the quality of the Intellectual Merit and Broader Impact activities,
including educational outreach, play an important role in funding decisions. If NSB approves a
project for future-year funding, it specifies its priority among all projects in the Board-approved
stage.1 If a project is not approved, or if an approved project’s plans are no longer deemed to
be clearly and fully construction-ready, NSB will remand that project to the Preliminary Design
phase for further work, or recommend that the project be terminated.
2.3.2.7 Inclusion in an NSF Budget Request
Each year, the NSF Director proposes, in priority order, the NSB-approved construction-ready
projects for the MREFC Account. If an MREFC “new start” is approved for inclusion in the
President’s Budget Request to Congress, then Congress may ask for additional information
through formal hearings and/or informal briefings. Once Congress passes an appropriations act
for NSF and the President signs it into law, NSF may request authority to obligate funds.

1

The Board assigns the very highest priority to projects that are under construction. There is no priority among active projects;
they should all move forward at a suitable pace.

Section Revision:
May 2, 2016

2.3.2-6

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.3 Final Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.3.3 Final Design Phase
2.3.3.1 Introduction – Final Design Phase
The goal of the Final Design Phase is to meet the requirements necessary to advance the
proposed project to the subsequent Construction Stage. Budgetary and administrative
requirements for entry include NSF review and approval of the project’s preliminary design as
described in the PEP, and NSB approval to include the project in a future NSF budget request.
Technical requirements include:








Delivery of designs, specifications and work scope that can be placed for bid to industry;
Refined bottom-up cost estimates and contingency estimates;
Implementation of a PMCS for project technical and financial status reporting;
Completion of recruitment of key staff and cost account managers needed to undertake
construction of the project;
Industrialization of key technologies needed for construction;
Finalization of commitments with interagency and international partners; and
Submission to NSF of a PEP1 for construction.

Successful exit occurs after the following steps are completed:
1.
2.
3.
4.

Successful review of the final design baseline including any receipt of bids;
Joint review by the DRB/MREFC Panel;
NSB review and approval for the NSF Director to obligate construction funds; and
Final negotiation of the terms and conditions of the award instrument for the activities
in conformance with the final baseline.

2.3.3.2 Final Design Review (FDR)
Projects should continue to receive pre-construction development funds in order to produce a
Final Design, which includes the following elements:






1

A final construction-ready design;
Tools and technologies needed to construct the project;
A project management plan describing governance of the project, configuration control
plans, and plans for reporting technical and financial status, managing sub-recipients
and working with interagency and international partners;
A fully implemented PMCS, including a final version of the resource-loaded schedule and
mechanisms for the project to generate reports – using the Earned Value Management

Refer to Section 3.4 for details of the PEP.

Section Revision:
May 2, 2016

2.3.3-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.3 Final Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)











System (EVMS)1 – on a monthly basis and use them as a management tool. Path
dependencies, schedule float, and critical path are defined;
Updated budget and contingency, including risk analysis, presented in a detailed WBS
format accompanied by a WBS dictionary defining the scope of all entries;
An updated Educational Outreach and Broader Societal Impact plan (including the scope
of work, budget and schedule) that also includes the capital investment required to
meet the needs of the proposed Educational Outreach and Broader Societal Impact
plan;
A significant proportion of the budget based on externally provided information such as
vendor estimates or quotes, publically available supplier prices, and the like;
All necessary partnership agreements and MOUs;
Fit-up and installation details of major components and commissioning strategy;
Plans for Quality Assurance and Safety;
Updated operating cost estimates; and
Certification that all of the pre-construction planning topics, including those listed in
Section 3, are fully complete and determined to be adequate.

Due to the Federal appropriations process, there may be one or more years between the PDR
and the start of construction, which is predicated on successful completion of the FDR. During
this time the NSF will review the project at least annually to ensure that the total project cost
and basis of estimate (BOE), acquisition strategies, schedule, and risk management plan
presented at the PDR are still valid.
The PO is responsible for organizing and leading the FDR. The review is conducted according to
the same standards and with the same respective roles for the PO and LFO Liaison as described
previously for the CDR and PDR.
The scope of the FDR includes assessment of the technical and project-management
components of the proposed project. A review panel may provide an objective view of the
project and a critical evaluation of the plans and risks embodied in the proposed program as
the schedule permits. In consultation with the IPT, the IMP should continue to be assessed
annually by the Program Officer and updated as required to ensure that the underlying
assumptions about the project remain valid. If construction funds fail to be appropriated as
planned, the NSF Director may choose to mandate annual project status reviews to assure NSF
of the continued viability of the project’s plan and budget for construction.

1

During construction, progress should be tracked and measured using the Earned Value method (this method is required by
OMB in Planning, Budgeting, Acquisition, and Management of Capital Assets, OMB Circular No. A–11 (2014). A discussion of
Earned Value is included section of Section 8, Guidelines for Earned Value Management.

Section Revision:
May 2, 2016

2.3.3-2

Large Facilities Manual: NSF 16-0XX (May 2016)
2.3.3 Final Design Phase
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Following the review, the PO and the LFO Liaison will each independently assess the review,
confer on areas of concern, share their views, and report their observations through their
respective supervisory chains.
In the event the project’s construction plans are determined to be inconsistent with the
pending budget request, NSF will undertake remedial action. Should remedial action be
necessary following the review, the sponsoring Directorate recommends this to the OD after
consultation with the IPT, internal deliberation, and if appropriate, consultation with the
MREFC Panel. Remedial action may include, for example, revision of the project’s budget,
scope, and/or schedule, or withdrawal of NSF’s request for construction funding (off ramp).
2.3.3.3 Exit from the Final Design Phase
Following a successful review of the final design baseline, the Director recommends to NSB that
it approve construction award(s).1 NSB reviews the recommendation and authorizes the making
of the award(s). Following this approval, an award instrument, generally a CA (s), between NSF
and the recipient institution(s) is negotiated, and construction activities begin in conformance
with the final baseline.
Following the approval to obligate funds and as part of NSF’s final negotiation of the award
instrument, the award-level NSF Cost Analysis will be initiated and conducted jointly with key
assurance members of the IPT; namely the Division of Acquisition and Cooperative Support
(DACS), DIAS, and the LFO. The analysis will encompass such things as negotiated sub-awards
and sub-contracts associated with initiating construction and negotiation of final indirect cost
and labor rates. The Cost Analysis will be conducted following NSF internal Standard Operating
Guidance.

1

See Appendix B for documentation required for recommendations.

Section Revision:
May 2, 2016

2.3.3-3

Large Facilities Manual: NSF 16-0XX (May 2016)
2.4.1 Construction Award Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.4

CONSTRUCTION STAGE

2.4.1 Construction Award Management and Oversight
After Congress appropriates funds for an MREFC project, NSF proceeds to award the
cooperative agreements (CAs) or contracts for construction of the facility. The policies and
procedures in the publically available NSF Proposal and Awards Policy and Procedures Guide,
and in the internal document NSF Proposal and Award Manual (PAM), apply to MREFC projects.
The PAM (available to the PO) covers the internal award process from proposal generation
through merit review, DRB and NSB reviews, and final award. The recipient(s) provides periodic
financial and technical status reports to NSF according to the terms and conditions of the CA or
contract. The project is subjected to periodic post-award reviews that may examine any or all of
the following topics: technical performance, cost, schedule, and management performance.
These reviews are typically held at the facility and are conducted at least annually. More
frequent reviews may take be scheduled based on the project’s expenditure rate or due to any
other technical or management issues that arise.
NSF selects the annual review panel members who are typically external experts covering all
aspects of the project, and assess technical progress, cost, schedule, and management
performance. These panels report directly to NSF and provide advice on project direction and
any needed changes. The reviews are organized and conducted by the PO in consultation with
the LFO Liaison. The PO has overall responsibility for organizing the review, and throughout the
review process acts as the interface between the NSF and the recipient. The PO authors the
review charge and organizes the review panel. The LFO Liaison strengthens the review process
by specifying language for incorporation within the charge and for aspects of the review agenda
pertaining to project management issues, and recommending panelists able to advise NSF in
non-science related areas of the review. Following the review, the PO and the LFO Liaison will
each independently assess the review, confer on areas of concern, share their views, and report
their observations through their respective supervisory. (Note: Many projects invite panels of
experts to review and advise on project plans and progress. Such panels report to the Project
Director, and are not a substitute for NSF-organized external oversight reviews.)
Generally, when cost and/or schedule performance begin to deviate from plans, change control
is exercised by the project through a Change Control Board (CCB)1 action, resulting in
modifications to the project’s budget or schedule contingency. It is also normal practice for a
project to update its budget and schedule Estimates to Complete (ETC), which also may result in
baseline changes.

1

A CCB comprises the senior project managers responsible for defining the project's resource requirements and allocating or
expending those resources. It typically consists of the Project Director, Project Manager, Business Manager, cost account
managers of principal work breakdown structure elements, chief scientist and engineer, and systems engineer. It may include
other project staff whose authority pertains to the range of activities considered by the Board.

Section Revision:
May 2, 2016

2.4.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.4.1 Construction Award Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Whenever a project approves a change control action that results in allocating or returning
contingency to the pool of contingency funds, the Performance Measurement Baseline (PMB)
budget may also change.
Similar change-control actions affect the PMB schedule. They revise the baseline project
schedule and the available schedule contingency or “float” time – that is, the difference
between milestones on the schedule’s critical path and the expected completion dates for
activities that lead to the accomplishment of those milestones.
Modifications to the performance baseline that are within the defined scope and do not change
the project end date or total project cost are referred to as “re-planning”. Re-planning may be
due to adjustments or re-organization of the project plan and/or may signify that contingency
funds are being expended in an expected manner. If the allocations of budget and schedule
contingency are below the budget or schedule thresholds identified in the CA between NSF and
Recipient, the change requests are approved unilaterally by the project. NSF approval is
required when the CCB recommends re-planning change actions that exceed the budget or
schedule thresholds identified in the CA between NSF and Recipient. Each will have a different
threshold for approval. Approval levels for scope changes are generally outlined in the CA.
It is essential for the project management to respect the project baseline rigorously,
maintaining each adjusted baseline in the project’s database along with the attributed CCB
actions. This allows the project and NSF to systematically track the evolution of the baseline
from its initial release through all subsequent changes.
“Re-baselining” occurs when the changes involve:
1. Increases in the NSB-approved TPC,
2. An extension beyond the approved end date, and/or
3. Major changes in scope.
When the proposed changes reach the re-baselining level, the approval process involves NSF
and may involve the NSB. Changes in end date follow NSF’s No-Cost Extension (NCE) policies.
An increase in TPC exceeding 20 percent of the NSB-approved baseline cost or $10 million
(whichever is smaller) must be reviewed and approved by the NSB following a recommendation
by the MREFC Panel and the Director.1 Prior to requesting approval to re-baseline, a new
external baseline review should be conducted to examine the nature of the problems
encountered, and to determine whether de-scoping should be exercised per the approved
scope contingency plan in the PEP or, if not, whether the problems can be solved by use of
budget contingency or other means. Upon review and approval, cost and schedule are
stabilized, and the contingency adjusted to an appropriate level.
1

Open Session Approved Minutes, 418th Meeting of the National Science Board, Resolution NSB-11-1, Adopted February 16,
2011.

Section Revision:
May 2, 2016

2.4.1-2

Large Facilities Manual: NSF 16-0XX (May 2016)
2.4.1 Construction Award Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Whenever significant cost increases are foreseen, it is most important that the LFO Liaison is
consulted early, concurs with the PO on the details of the Originating Organization’s plan, and
advises and concurs on details of the external re-baselining review. Similarly, when there are
indications that the project contingency will fall below reasonable standards,1 the PO should
discuss plans for dealing with the variance with the Project Director. This information should be
clearly noted in the monthly status report that goes to the HLFO. The LFO is a resource for
helping to deal with such problems and for helping to identify steps that can be taken to restore
adequate contingency.
In addition to supplying regular status reports required in the terms and conditions of the CA, it
is essential that project staff inform the PO and/or the G/AO in a timely manner of major issues
or significant changes in project status, such as a potential re-baselining, problems with
partnerships, or surprising research and development results. NSF management, the MREFC
Panel and the NSB should in turn be informed of such developments by the PO. The primary
mechanism for coordinating both the transfer of information and the coordination of any
required actions by NSF is through the NSF Integrated Project Team (IPT).
On rare occasions, MREFC projects under construction may encounter unforeseen budget or
programmatic challenges that are of a substantial enough level to be considered grounds for
termination or significant modification to the original project goals. NSF will provide the NSB
with appropriate information and a recommendation via the MREFC Panel and the Director.
The NSB will decide whether termination or significant modification to the original project goals
is warranted.2

1

See details in Section 4.5, Requirements for Performance Oversight, Reviews and Reporting.

2

Joint NSB-NSF Management Report: Setting Priorities for Large Facility Projects Supported by the National Science Foundation
(NSB-05-77); September 2005.

Section Revision:
May 2, 2016

2.4.1-3

Large Facilities Manual: NSF 16-0XX (May 2016)
2.4.2 Commissioning Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.4.2 Commissioning Plan
The transition from construction to operations is rarely abrupt. Many facility projects require a
testing and commissioning phase, funded through the MREFC Account. The scope of these
activities is defined in the PEP and included in the initial MREFC budget request as part of the
performance baseline. The PEP is included by reference in NSF’s CA or contract with the
Recipient institution, documenting the mutual understanding of the work scope funded by
MREFC funds. In some cases, particularly with distributed facility projects, early operations
funding begins to increase as aspects of a facility come on line, although full construction
funding may not have concluded. Although these phases overlap in time, they must be
budgeted and managed separately due to segregation of funds requirements.
NSF will ask for a commissioning plan at least one year prior to initial commissioning activities.
The scope of commissioning work is to undertake initial operation of the facility and bring it up
to the design level of operation in accordance with the IMP. The IMP is updated prior to the
operations stage to define reviews, decision points, strategies for renewal or recompetition,
plan for advanced R&D or technology refresh, upgrades, etc.
The content of the commissioning plan will be adapted to the specific nature of the facility, but
at a minimum it should include:












A detailed bottom-up cost estimate for operations.
A detailed management plan for operation of the facility, including the roles of key staff
and plans for advisory committees.
Education and outreach plans and their associated costs, including the scope to work,
associated budget and schedule.
The costs of an in-house research program, if applicable.
A detailed set of acceptance criteria that establish that the facility is finished and ready
to commence routine operations.
A listing of which environmental safety and health (ES&H) standards will be followed by
the recipient and a description of how adherence to those standards will be verified. A
policy for reporting to the NSF accidents or environmental releases should also be given.
This may be given as a reference to an existing ES&H plan for the project.
A listing of which cyber-security standards will be followed by the recipient and a
description of how adherence to those standards will be verified. A policy for reporting
to NSF of any breaches of cyber-security should also be given. This may be given as a
reference to an existing cyber-security plan for the project.
A discussion of how major maintenance issues (such as budgeting for periodic
replacement of long-lived capital assets who useful life extends beyond the duration of
the CA) will be handled
A discussion and a preliminary cost estimate for decommissioning the facility.

Section Revision:
June 17, 2015

2.4.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.4.2 Commissioning Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)



A set of performance goals and metrics sufficient to establish that the facility is
operating successfully. These will be updated in each year’s Annual Work Plan (see
below).

Once the commissioning plan is complete, an Operations Readiness Review (ORR) will be held
to examine and comment on the plan. This can be considered as one of the required annual
reviews. The review is organized and conducted by the PO in consultation with the LFO Liaison
similarly to other reviews described above.

Section Revision:
June 17, 2015

2.4.2-2

Large Facilities Manual: NSF 16-0XX (May 2016)
2.4.3 Construction Award Close-out
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.4.3 Construction Award Close-out
2.4.3.1 Project Close-out Process
(Intentionally left blank – this section to be written)
2.4.3.2 Request for No-Cost Extension
Under NSF policy, the Program Officer (PO) has the authority to approve the first No-Cost
Extension (NCE). However, the PO will generally work closely with members of the NSF
Integrated Project Team (ITP) to ensure the request meets the requirements for large facility
projects as described herein. Any subsequent NCE’s must be approved by the Grants and
Agreements Officer (G/AO) who is also an integral member of the IPT. As the project nears
completion, close-out activities will become a discussion item for the IPT.
Only tasks within the approved project scope may be included in the NCE. As stated in Section
4.2.5, Budget Contingency Planning for the Construction Stage, any unused funds (either
contingency or positive cost variance) must be returned to the agency.
Many intended tasks will already be clearly contained within the approved project scope and
can be directly associated with a particular WBS element. Tasks which cannot be found to fall
within an approved WBS element will be allowed only after they have been reviewed and
approved as new scope through the change and/or configuration control processes contained
in the Project Execution Plan. Depending on the magnitude, this may require very high level
approvals within the agency. It is highly recommended that the discussion of scope, and the
ability to assign to an approved WBS element, takes place prior to the NCE request.
Good practice suggests that all other project tasks, i.e. those not included in the NCE request,
should be closed out by the original project end date. This means that all risks and liens for
those tasks are also closed out, and that no funds are carried forward for remediation of
problems that arise in the future. The close-out of completed tasks also allows for a more
precise calculation of remaining cost variance and/or contingency which facilitates good
decisions making on the part of the Project and NSF. If any tasks slated for close-out are not
completed by the original end date, then NSF must be notified that the tasks will be carried
over into the extension period as part of the NCE request.
It is anticipated that the list of tasks to be performed during the extension may change with
time as final negotiations and decisions are made and actual costs are realized. Some tasks may
be held back and subsequently removed as scope contingency options when available
resources or priorities change. Other tasks within the approved scope of the project may be
added (for example, as a result of a reprioritization exercise following final acceptance reviews
or because they are delayed past their close-out dates). Tasks may be added or removed from
the list with adequate justification and with the written approval of NSF. All final close-out
documentation will be saved to the official record by the PO.
Section Revision:
June 17, 2015

2.4.3-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.4.3 Construction Award Close-out
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Written requests for NCEs should be submitted to the PO and should include the information in
the following list:
1. List of the tasks to be completed during the extension period and justification that they
are within project scope.
a. Link the tasks to the associated WBS element and give a short justification of
how they fit within existing project scope. Risk mitigation effort should be
associated with an identified and documented risk element.
b. Provide the total burdened estimated cost for each task. Detailed cost estimates
do not have to be provided, but should be documented and available if
requested.
c. The justification for each task will typically fall into one or more of several
categories: (1) open purchase orders and invoices associated with items whose
delivery is delayed beyond the current period of performance, for example due
to subcontractor performance, (2) rework of existing tasks within the approved
scope, for example due to workmanship or performance issues, (3) existing tasks
within the approved scope that have not yet been completed, and (4) risk
mitigation to address in-scope performance issues. An example of a task list with
justifications is given in the sample Table 2.4.3-1 on the following page.
2. Indication of which tasks provide scope contingency options1 if resources (time, staff,
budget, etc.) become limited. Briefly indicate why each task is a candidate for descoping and give any deadlines for exercising the de-scope option. NSF must be notified
when and if the scope contingency option is taken and tasks are removed from scope,
including the impact on project deliverables or performance, if relevant.
3. Description of what funds will be used to cover the proposed tasks – remaining
contingency, unexpended baseline budget, positive Cost Variance, partner funds, etc.
Give the project performance baseline ETC with all tasks included and compare to
remaining contingency and TPC. State a confidence level for completing all work within
budget, including the use of any scope contingency options. Indicate if any tasks involve
already obligated funds and give the amount of those funds.
4. Summary schedule or schedule highlights of the extended tasks, including significant
milestones and the new end date. Provide (BOE) for the new end date, including
schedule contingency, and give a confidence level for completing by that date.

1

Scope contingency is defined in Section 5.2.3.

Section Revision:
June 17, 2015

2.4.3-2

Large Facilities Manual: NSF 16-0XX (May 2016)
2.4.3 Construction Award Close-out
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 2.4.3-1

Task #

Sample of a No-Cost Extension Tasks Table
Burdened
Subtotals ($K) WBS

Task Description

Justification

1

Modifications to electronics
control boards

40.5

3.7
Environmental
Systems ADCs

Rework of existing in-scope
task; technology not
performing as intended

2

Delivery of 3 cryo-pumps

114.9

4.2 Vacuum
Systems

Existing in-scope task; Late
delivery on open contract with
obligated funds

3

General purpose utility carts

25.8

2.4.5 Monitoring
and Maintenance
Equipment

Existing in-scope task; Late
delivery; 1 unit added based
on revised needs estimate

4

Vendor contract to test
relationship of performance
versus temperature on
sample size widgets

32.4

5.2.3 Sys Eng:
Integrated
testing

Risk mitigation added to
address in-scope performance
issues for integrated systems.
Risk Register ID #14-31.

5

Labor extensions for project
management and business
offices

184.2

1.2 Project
Controls

Existing in-scope task; revised
effort, salary, and overhead
estimates, including escalation

TOTAL ($K)

Section Revision:
June 17, 2015

$397.8

2.4.3-3

Large Facilities Manual: NSF 16-0XX (May 2016)
2.5.1 Operations Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.5

OPERATIONS STAGE

2.5.1 Operations Management and Oversight
Although NSF does not directly manage the operations of the facilities it supports (with the
exception of Antarctic activities), the agency engages in oversight and assurance of facility
awards during each stage of the facility’s life cycle. In oversight, NSF employs a team-oriented
approach in which scientific and engineering staff work closely with business operations staff.
Additional detail on facility operations may be found in Section 3.5 of this manual1 and among
the special topics found in Section 4, Key Management Principles and Requirements for Large
Facilities.
The recipient responsible for construction or acquisition of a new facility is normally the entity
that submits a proposal for operation of the facility during the construction stage. However, the
Operations Stage may be managed by a different entity, depending on circumstances stated in
the IMP.
The operations proposal is merit-reviewed following NSF’s guidelines. Operations activities are
funded through NSF’s R&RA and/or Education and Human Resources (EHR) account. Testing
and acceptance, user training and engineering studies occur as the facility transitions to full
operation. Operations include the day-to-day work required to: support and conduct research
and education activities; ensure that the facility is operating efficiently and cost-effectively; and
provide small- and intermediate-scale technical enhancements when needed to maintain stateof-the-art research capabilities.
Given the long operations stage of most large facilities, upgrades and refurbishment of
equipment may be required over time in order to stay at the research frontier. In the case of an
observatory, this may include new instruments and cameras. For a sensor network, it may
include the deployment of additional sensors or renewal of cyber-infrastructure. At an
accelerator facility, the upgrades may take the form of higher energy or luminosity or new
detectors. In general, these upgrades and renewals will be funded from R&RA funds, either
from a portion of the operating funds designed for such purposes or from separate equipment
and instrumentation programs. Funding for more significant upgrades (if they exceed the
MREFC threshold) may come from the MREFC account. In that case, the approval process is the
same as that for a new MREFC project.
Three key aspects of NSF oversight and assurance of large facility operations, which (if
applicable) are referenced in and required by the CA, are: (1) Annual Work Plans, (2) Annual
Reports, and (3) Annual Operations Reviews. NSF or the cognizant agency may also conduct
periodic audits.

1

These sections are in preparation.

Section Revision:
June 17, 2015

2.5.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.5.1 Operations Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Annual Work Plan
The Annual Work Plan describes what the facility expects to accomplish in the coming fiscal
year. The Annual Work Plan should include a series of high level performance goals (clear and
agreed upon goals and objectives, performance measures and, where appropriate,
performance targets) for the coming year. The goals should include both scientific and
operations issues (i.e., installation of new equipment or commissioning of new buildings,
maintenance, Education and Oversight Training [EOT] and ES&H). The goals will naturally vary
from facility to facility and should be agreed upon between the recipient and the NSF Program
Officer (PO). Goals in the Annual Work Plan should meet the standard of being specific, realistic,
measurable and time-based. The LFO Liaison will review the goals to ensure they meet this
standard.
Annual Report
The Annual Report describes in detail the activities of the facility in the previous twelve months.
This report is required to review progress on that year’s performance goals (as described in the
Annual Work Plan). Due to changing research priorities or external forces not all performance
goals may be met each year but an explanation of progress on each goal is required. The PO
reviews and approves the Annual Report.
Annual Operations Reviews
In most cases, NSF will annually conduct Operations Reviews of its major multi-user research
facilities, utilizing an external panel of experts spanning the principal range of functions
necessary to sustain facility operations, or carry out or participate in an alternate activity that
accomplishes an equivalent purpose. Exceptions to the annual review (or its alternate) occur
when NSF partners with other entities to fund operations. In those instances, the MOU
between the partners defines the process for monitoring: (1) identification and
accomplishment of programmatic goals; (2) fiscal accountability; (3) stewardship of NSF assets;
and (4) compliance with laws and regulations. These reviews (or their alternates) should
determine the extent to which the facility is meeting the goals of its Annual Plan, discuss any
upcoming challenges for operations, and highlight best practices that could be applied to other
large NSF facilities. Metrics and performance goals or targets should include objectives related
to educational outreach and broader societal impacts, in addition to research goals of the
operating facility. Whenever possible, the review should be conducted at the facility itself by an
external panel comprised of experts in the operations of similar large scientific facilities and
representatives of the user community served by the facility. The panel should produce a
formal written report. Results of the review are used by NSF to provide feedback to the facility
operator in the formulation of goals or targets for the coming year. (The Operations Review is

Section Revision:
June 17, 2015

2.5.1-2

Large Facilities Manual: NSF 16-0XX (May 2016)
2.5.1 Operations Management and Oversight
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

not meant to compete with the Business Systems Review1 (BSR) which looks at business
processes.)


The review is organized and conducted by the PO in consultation with the LFO Liaison.
The PO has overall responsibility for organizing the review (or representing NSF’s
interests in the case of a partnership), and for acting as the interface between the NSF
and the project’s proponents throughout the review process. The LFO Liaison advises
the PO during the planning and execution of the review to ensure that there is
consistent practice across NSF in the formulation of performance goals, that goals and
objectives are clearly stated and represent quantifiable performance measures or
targets where practical, are periodically reported, and that an evaluation and feedback
mechanism is implemented as an essential part of an ongoing program of continual
performance enhancement.



Following the review, the PO and the LFO Liaison will each independently assess the
review, share their views, confer on areas of concern, and report their observations
through their respective supervisory chains.



In most cases, observers of the review shall include the Program Officer, the Grants and
Agreements Officer, the LFO Liaison and other staff from the Large Facilities Office, and
possibly other NSF staff from the Integrated Project Team. Budget considerations,
logistical constraints, or alternate processes for review agreed to by NSF and its funding
partners may result in exceptions to the number and range of NSF staff participating.

1

See Section 4.5.3.3 for discussion of the BSR process as well as the NSF BSR Guide. To avoid duplication of effort, the scope of
the BSR is adapted to utilize relevant information stemming from other reviews and audits.

Section Revision:
June 17, 2015

2.5.1-3

Large Facilities Manual: NSF 16-0XX (May 2016)
2.5.2 Renewal/Recompetition
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.5.2 Renewal/Recompetition
Most NSF facilities will be operated by a managing organization. Because facility lifetimes are
long (some current facilities have operated in excess of 40 years), recompetition of
management is appropriate at intervals. Whenever practical, NSF seeks to make competitive
renewal awards for operation of large facilities after external merit review. See Section 3.5.2 for
procedures for Renewal and Recompetition. The NSB issued a statement requiring full and open
recompetition of awards for operation of major facilities upon their completion and after an
appropriate time period to bring the facility to sustainable operations.1 The goal of competition
is to stimulate new approaches toward more effective management that may offset any
potential for increased costs, and ideally may achieve some cost savings. Important
considerations beyond performance of current management include how recompetition might
affect the scientific productivity of the facility and the burden it would place on the community.
Even in cases where the existing management has been explicitly and rigorously reviewed and
found to be effective, the benefits of competition may outweigh any short-term disadvantages
of recompetition. The determination of whether to compete the effort is based on the expert
advice of NSF staff and, where applicable, external sources using the facility, and should be
presented to the NSB for approval.

1

See NSB Statement on Competition, Recompetition, and Renewal of NSF Awards, NSB 08-16,
https://www.nsf.gov/nsb/publications/2008/nsb0816_statement.pdf.

Section Revision:
June 17, 2015

2.5.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.6 Divestment Stage
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.6

DIVESTMENT STAGE

To remain at the research frontier and support new facilities, NSF will consider retiring existing
facilities when the science they enable is of a lower strategic priority than science that could be
enabled by alternate use of the funds. Such decisions will be difficult to make, in part because
of the number of stakeholders and interested parties, and will require extensive community
consultation and input, which may come from “blue ribbon” panels, National Academies
committees and professional societies. In some cases in which a facility can continue to be
productive, it may be possible to transfer ownership to another agency, a university or a
consortium of universities. It is the responsibility of the Directorates and Divisions to
periodically review their facilities portfolio and to consider which facilities may have reached an
appropriate end of NSF support.
When the decision is made to close or transfer ownership of a facility, a transition plan will be
developed, which includes all divestment costs and liabilities, including disposal of equipment,
environmental and site remediation or restoration, pension and health care responsibilities,
etc.

Section Revision:
May 2, 2016

2.6-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.7.1 Flexible Requirements for Non-MREFC Facility Projects
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.7

APPLICATION OF MREFC PROCESS TO NON-MREFC FUNDED PROJECTS

2.7.1 Flexible Requirements for Non-MREFC Facility Projects
The project management processes and principles described in the preceding section are
generally applicable to all large facility projects, irrespective of the source of construction
funding. However, considerable flexibility is allowed in the management approach to adapt the
process to the requirements and scope of any particular project.
This section provides guidelines for planning and managing new facility projects that are not
constructed with funding from MREFC accounts.1 This is usually the case when the project does
not qualify for MREFC funding2 and/or the sponsoring Directorate or Office chooses not to
apply for MREFC funding.
This section applies to non-MREFC facility projects that take a multi-stage design approach
similar to that described in Section 2, and that are large enough to require multiple levels of
approval within NSF beyond the level of the Originating Organization. (It does not pertain to
awards for centers, or other types of awards unrelated to facilities which require approval
merely because of their large size.) The total cost of a non-MREFC facility project generally
ranges from millions to tens of millions of dollars or more. The majority of these projects will
require NSB award approval.3
Non-MREFC projects are not subject to the same requirements for Conceptual Design,
Preliminary Design, and Final Design reviews outlined in Section 2. Nor are they required to use
the three sets of ranking criteria in Appendix A or subject to review by the MREFC Panel.
However, the elements described in Section 2 make a useful toolkit for a Directorate or Office
to use in planning and managing all large facilities that proceed through these design stages.
How the elements might apply is the focus of this section.
As in the case of MREFC projects, NSF is committed to the principle that flexibility does not
preclude rigor. For projects that do not require a multi-stage design approach, the PO should
explain the variation and define the management approach taken in the project’s IMP.

1

R&RA (and possibly EHR) appropriations accounts are used to support the construction of non-MREFC large facilities. In
addition, non-construction activities of MREFC-funded construction projects, including research, design, development, and
operations costs, are normally funded through the R&RA and/or EHR appropriations accounts.
2

See the previous Section 2.1.2 for eligibility requirements for MREFC funding.

3

See the description of NSB roles and responsibilities in Section 2.1.6.

Section Revision:
May 2, 2016

2.7.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
2.7.1 Flexible Requirements for Non-MREFC Facility Projects
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Selection Criteria: Both MREFC and non-MREFC facility projects should depend on a proposaldriven process with external and internal merit review. Other factors to consider might include:








Exceptional opportunity to enable frontier science and engineering (S&E) research and
education;
Urgent contemporary research and education need;
High priority within the relevant S&E communities;
Accessibility to an appropriately broad user community;
Partnerships well defined;
Technical feasibility and risks thoroughly addressed; and
A well-developed PEP.

PO Oversight: At the earliest practical point, each large-facility project is assigned an NSF PO1
with primary responsibility for award management and project oversight. As noted in Section 2,
NSF restricts the choice of POs overseeing MREFC-funded activities to permanent NSF
employees2 to assure continuity of oversight. POs overseeing non-MREFC funded projects are
exempt from this statutory requirement. However, the principle should be taken into
consideration for non-MREFC projects by matching the term of assignment of the cognizant NSF
oversight staff to the duration of the late-stage planning and construction activity.
Alternatively, assigning a team of POs with a mix of permanent and rotating staff may help
ensure continuity.
Large Facilities Working Group: The Large Facilities Working Group3 is available to review and
provide comments on the IMP for a large facility project, independent of the source of
construction funds.
Interaction with Head, Large Facility Office: The HLFO is available in an advisory capacity to
NSF staff working on non-MREFC funded projects as a resource for best practices for project
management and business oversight. But the HLFO’s involvement is not mandatory unless so
directed by the Director, the Deputy Director or the AD/ Office Head of the Originating
Organization(s). The HLFO may be asked by the Director or Deputy Director to review DRB and
NSB packages for non-MREFC facilities.
Integrated Project Teams: At the earliest opportunity, the originating Directorate or Office
should coordinate with the PO to organize an IPT4 to provide advice and help coordinate NSF’s
oversight and assurance.

1

Also referred to within NSF as Program Director or Program Manager.

2

See Public Law 107-368, Section 14(c).

3

The functions of the Large Facilities Working Group are described in “Roles and Responsibilities” in Section 2.1.6.

4

Ibid.

Section Revision:
May 2, 2016

2.7.1-2

Large Facilities Manual: NSF 16-0XX (May 2016)
2.7.1 Flexible Requirements for Non-MREFC Facility Projects
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

NSB Budget Approval: Unlike MREFC projects, non-MREFC projects do not require formal NSB
approval as part of the budget process in order to be included in future NSF budget requests.
Rather, the non-MREFC projects are considered by NSB in the course of reviewing the entire
NSF budget request. However, both MREFC projects and non-MREFC facility projects above the
NSB approval threshold require both DRB and NSB approval before an award is made.
NSF Office of the Director: Providing information early in the planning process to the Office of
the Director is advisable. The Director may wish to share information items periodically with
the NSB.

Section Revision:
May 2, 2016

2.7.1-3

Large Facilities Manual: NSF 16-0XX (May 2016)
2.7.2 Pre-construction Planning and Development of Non-MREFC Projects
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.7.2 Pre-construction Planning and Development of Non-MREFC Projects
As is the case with most MREFC projects, pre-construction planning and development of nonMREFC facility projects may progress through sequential stages of increasing investment,
planning, assessment and oversight. At each stage, the technical evolution of the project and
NSF’s preparatory planning and budgeting are coordinated and synchronized to achieve an
orderly evaluation process that results in eventual construction funding for the most
meritorious projects.
The sponsoring Directorate decides upon the appropriate degree of rigor and formality in preconstruction planning necessary to ensure that the project is well defined and appropriately
budgeted. These decisions are based upon the size and complexity of the proposed project, and
are documented and justified in the project’s IMP.
As with MREFC projects, most non-MREFC funded projects begin when NSF responds to a
community initiative (exceptions may include infrastructure replacement and/or addition). Such
initiatives may take different forms – for example, a report from a community planning activity
or a formal proposal. The sponsoring Directorate or Office’s decisions and strategies for project
review, funding and oversight are delineated in the IMP. The IMP specifies how NSF will
supervise management of a project, and provides budgetary estimates for developing,
constructing and operating the facility. It also identifies divestment liabilities and lays out a
strategy for financing these activities as well as the concomitant NSF oversight requirements.
The PO in the sponsoring Directorate or Office prepares the IMP in the early stages of the
project’s conceptualization. It is reviewed and approved by the AD/Office Head of the
Originating Organization(s). The Originating Organization(s) may design and adopt oversight
processes and procedures that are flexibly tailored to the needs of the particular project.
Very large or complex projects will require more formalized pre-construction planning and
frequent status reporting. Smaller projects will have appropriate requirements.1 The project
management approach used must be scaled to the needs of a particular project. For example,
project management controls used to manage project resources, document project activity and
plan alternate courses of action to mitigate risk will be much more sophisticated and costly for
a large-scale project than for a small one.
Budgets, schedules, risk assessments, and project management plans will be similarly scaled.
The IMP defines NSF’s expectations for the appropriate level of scaling that optimally matches
oversight requirements to project needs. NSF conveys these expectations to the project
proponents for incorporation in their PEP as appropriate.

1

Refer to Section 3.3 “Guidelines for Development of Internal Management Plans for Large Facilities” (an internal NSF
document) for a description of the intent and content.

Section Revision:
May 2, 2016

2.7.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
3.1 Introduction to Management Plans
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

3
3.1

LIFE CYCLE MANAGEMENT PLANS FOR LARGE FACILITIES
INTRODUCTION TO MANAGEMENT PLANS

Section 3 contains descriptions and guidelines for creating the plans that NSF and Recipients
use in the management and oversight of Large Facilities. They include two plans produced by
NSF and three plans that are the product of the facility designers, constructors, and operators.
The NSF Facility Plan, as described in Section 3.2, is a yearly exposition of the status and
intentions for the NSF portfolio of existing and candidate MREFC facility projects, in the context
of the current climate of science opportunities and priorities. It also lays out the objectives and
compelling needs for major facilities, given the frontier research opportunities of the time. The
NSF Facility Plan informs decision-making in the Executive Branch and Congress, as well as
serving as a vehicle for communication with scientific communities. It is available to the public
on the NSF web site.
Section 3.3 describes and points to an internal NSF document with guidelines on creating an
Internal Management Plan (IMP), the NSF document that captures how NSF will oversee awards
for large facilities throughout the life cycle, from candidate MREFC facility projects in design,
through construction and operation, and ultimately, through divestment. An IMP also provides
financial strategies for funding given the estimated budgetary estimates. Both the guidelines
and the created IMPs are internal NSF documents.
The Project Execution Plan (PEP) is produced by the Recipient to detail how management and
execution of design and construction of a major facility will be accomplished. The PEP advances
in maturity from a rudimentary form required at the Conceptual Design Review to a fully
mature document ready to support construction at the Final Design Review. Section 3.4
provides a list of the required components of a PEP and guidelines for creating those
components.
Operations Plans are addressed in Section 3.5, including timelines for submission and review of
operations proposals from prospective Recipients and guidelines for content of proposals and
plans. Operation Plans cover all aspects of operations, maintenance, upgrades, and research
and education programs. Guidelines are also given for the procedures for renewal or
recompetition of an award for an operating facility.
Guidelines for plans to terminate operations under NSF awards are in development, with
Section 3.6 provided as a placeholder. Divestment, termination of NSF funding and oversight of
a facility, may be accomplished through transfer to another agency or funding source or
through decommissioning and deconstruction.

Section Revision:
May 2, 2016

3.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
3.2 NSF Facility Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

3.2

NSF FACILITY PLAN

The NSF Facility Plan, which is updated annually and publicly available, serves as valuable
planning tool both within and outside NSF. It also provides a comprehensive exposition of
needs and plans to inform decision-making in the Executive Branch and Congress, and serves as
an important vehicle for communicating with research communities.
The first section of the Facility Plan provides an extensive discussion of the frontier research
objectives and opportunities that provide the context and compelling need for major facilities.
The contents of this section derive from workshops, advisory committees, National Research
Council (NRC) reports, expertise of visiting and permanent scientific staff, and unsolicited
proposals from the community. The Facility Plan’s second section provides annual updates on
the status and progress of each Major Research Equipment and Facilities Construction (MREFC)
project and candidate project. It also maps these projects against the objectives and
opportunities contained in the first section. In particular, this section addresses:






Preliminary Design/Readiness Stage Projects – Projects in various stages of readiness,
including those that will be ready to go the National Science Board (NSB) for approval
within approximately the next year, and those that the MREFC panel has recommended
for advancement to the Preliminary Design Phase.
NSB Approved Projects – Projects that the NSB has approved for funding in a future
budget request.
Possible New Starts – Facilities for which initial MREFC funding is requested in NSF’s
annual budget request.
Ongoing MREFC Projects – Facilities already in operation or under construction.

In addition to providing regular status reports, the Facility Plan reflects the Administration’s
priorities for new start projects, NSB priorities for NSB-approved projects, and the NSF
Director’s priorities for projects in the Preliminary Design Phase. Ongoing MREFC projects are
always given the highest budget priority.
Every year new science and engineering opportunities arise and new priorities assert
themselves. As a result, no roster or ranking of potential MREFC projects is ever final.
Responsible stewardship of public funds demands that all candidate efforts be evaluated and
reevaluated constantly in the context of the latest, most pressing research goals and the most
profoundly important unanswered questions.
It is the responsibility of the Head, Large Facility Office (HLFO) to develop and maintain the
Facility Plan. The plan is approved by the Director and submitted to the NSB in March of each
year.

Section Revision:
May 2, 2016

3.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
3.3 Internal Management Plans for the Large Facility Life cycle
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

3.3

INTERNAL MANAGEMENT PLANS FOR THE LARGE FACILITY LIFE CYCLE

Please contact the cognizant NSF program officer for details, which are given in the internal NSF
document Guidelines for Development of Internal Management Plans for Large Facilities.
This document provides guidance to the PO on topics to be included in an Internal
Management Plan (IMP), grouped by life-cycle stage. The IMP is the primary document that
describes how NSF will oversee development, construction, operation and eventually
divestment and termination of support for a major facility. The requirement to develop an IMP
is described in Section 2.3.1 for MREFC and in Section 2.7 for non-MREFC projects. Two primary
purposes are served by development of an IMP:



It defines in specific detail how NSF will conduct oversight of a project; and
It provides budgetary estimates for developing, constructing and operating the facility,
identifies divestment liabilities, and lays out a strategy for financing these activities as
well as the concomitant NSF oversight requirements.

Section Revision:
May 2, 2016

3.3-1

Large Facilities Manual: NSF 16-0XX (May 2016)
3.4.1 Components of a Construction-ready Project Execution Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

3.4

PROJECT EXECUTION PLAN

3.4.1 Components of a Construction-ready Project Execution Plan
Essential components of a construction-ready Project Execution Plan (PEP), common to most
plans for construction of large facilities, are listed in Table 3.4.1-1 below, as an example of the
extensive nature of the pre-construction planning that should be conducted prior to expending
MREFC funds to execute the project. Additions or alterations to this list are likely, due to the
unique nature of each specific project. While many of the listed topics cannot be substantively
addressed at the earliest stage of project planning, it is important that project advocates are
aware, at the outset, of the full scope of pre-construction planning activities that should be
undertaken and the consequent pre-resources required. As the project matures through
Conceptual Design, Preliminary and Final design, these topics become correspondingly better
defined.
Table 3.4.1-1

List of the Essential Components of a Project Execution Plan, with Sub-Topics and Descriptions

Component

Sub-Topics

Description of Sub-Section Requirements

1. Introduction

1.1 Scientific Objectives

Description of the research objectives motivating the facility
proposal.

1. Introduction

1.2 Scientific
Requirements

Comprehensive statement of the Requirements Matrix/ Key
Science Requirements to be fulfilled by the proposed facility
(to the extent possible identifying minimum essential as well
as desirable quantitative requirements), which provide a
basis for determining the scope of the associated
infrastructure requirements.

1. Introduction

1.3 Facility /
Infrastructure

Description of the infrastructure necessary to obtain the
research and education objectives.

1. Introduction

1.4 Community
Outreach and Impacts

Description of the Educational Outreach and Broader
Societal Impacts associated with the purpose of the facility,
including the scope of work, budget and schedule related to
community or society related actions or interactions.

Section Revision:
May 2, 2016

3.4.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
3.4.1 Components of a Construction-ready Project Execution Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Component

Sub-Topics

Description of Sub-Section Requirements

2. Organization

2.1 Project Governance

Project Governance, showing Oversight and Advisory Plans
with clear lines of authority, responsibility, and communication between Internal and institutional governance and
oversight and advisory committees.

2. Organization

2.2 Project Organization

Project Organizational Structure, showing clear lines of
authority, responsibility, and communication between NSF,
any partners, and the Recipient.

2. Organization

2.3 Partnerships

Role of interagency or international partners in future
planning and development and/or construction. Plans,
agreements, and commitments for interagency and
international partnerships. Description of the project’s
stakeholders and their roles, responsibilities and meeting
schedules.

2. Organization

2.4 Roles and
Responsibilities

Roles and Responsibilities of key project personnel and
governance groups.

2. Organization

2.5 Community
Relations and Outreach

Community Relations and Outreach plans for building and
maintaining effective relationships with the broader
research community that will eventually utilize the facility to
conduct research. Description of scientific and educational
outreach programs.

3. Design and
Development

3.1 Project
Development Plan

Description of activities that will be undertaken in order to
achieve readiness for construction, such as design,
prototyping, manufacturing process validation, vendor
qualification, modeling and simulation, creation of required
project management plans, forming partnerships, etc.

3. Design and
Development

3.2 Development
Budget and Funding
Sources

Estimate of total budget required to perform Design and
Development, including NSF funding and any contributions
from partners and other outside sources.

3. Design and
Development

3.3 Development
Schedule

Schedule of design and development activities and
milestones, at a level of detail appropriate to the maturity
and complexity of the work.

Section Revision:
May 2, 2016

3.4.1-2

Large Facilities Manual: NSF 16-0XX (May 2016)
3.4.1 Components of a Construction-ready Project Execution Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Component

Sub-Topics

Description of Sub-Section Requirements

4. Construction
Project Definition

4.1 Summary of Total
Project Definition

Summary at Work Breakdown Structure (WBS) level II of
total construction project scope, cost, and schedule required
to complete the construction or implementation project,
indicating the Performance Measurement Baseline (PMB)
and contingencies funded by NSF as well as any associated
scope supported by other funding sources.

4. Construction
Project Definition

4.2 Work Breakdown
Structure (WBS)

WBS contains a product-oriented, hierarchical framework
that organizes and defines the total scope of the project into
individual project component that represent work to be
accomplished, aggregating the smallest levels of detail into a
unified project description. The WBS integrates and relates
all project work (cost, schedule and scope) and is used
throughout the project management to identify and monitor
project progress.

4. Construction
Project Definition

4.3 WBS Dictionary

WBS dictionary defining scope of each WBS element,
through all levels.

4. Construction
Project Definition

4.4 Scope Management
Plan and Scope
Contingency

The plan describes how the scope will be defined,
developed, monitored, controlled, and validated, and how
up-scoping opportunities and descoping options will be
realized. Scope Contingency complies savings from potential
de-scoping options, with decision points for exercising
options and time-phased cost and schedule.

4. Construction Project Definition

4.5 Cost Estimating Plan
and Baseline Budget

A plan describing how the cost estimating guidance in this
manual will be implemented, how the cost estimate will
evolve over time, and how the “Cost Model Data Set” will
meet the various needs of the project. The CEP should
typically include a narrative and sufficient detail explaining
the ground rules and assumptions, practices, systems, and
calculations used to develop the baseline budget for the
PMB, by WBS element.

4. Construction
Project Definition

4.6 Budget Contingency

Contingency budget and description of method for
calculating contingency, including confidence level for
completing within budget.

4. Construction
Project Definition

4.7 Cost Report, Cost
Model Data Set, and
Basis of Estimate

The Cost Book Report is the compilation of Cost Book
Sheets, typically used to present total project cost, but may
be used to present rolled-up costs for smaller elements or
sub-elements The cost model data set is used as input to
software tools and/or project reports to organize, correlate,
and calculate different project management information.
The Basis of Estimate provides supporting documentation
outlining the details used in establishing project estimates
such as assumptions, constraints, level of detail, ranges, and
confidence levels.

4. Construction
Project Definition

4.8 Funding Profile

Show the proposed NSF Funding Profile by year with
baseline commitment and anticipated contingency
allocation profiles. Also provide a total funding profile from
all sources if applicable.

Section Revision:
May 2, 2016

3.4.1-3

Large Facilities Manual: NSF 16-0XX (May 2016)
3.4.1 Components of a Construction-ready Project Execution Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Component

Sub-Topics

Description of Sub-Section Requirements

4. Construction
Project Definition

4.9 Baseline Schedule
Estimating Plan and
Integrated Schedule

Schedule (without contingency) for the overall project and
each major subsystem, including system integration,
commissioning, acceptance, testing and transition activities;
as well as major milestones and milestones for reviews,
critical decisions and deliverables. It uses formal scheduling
programs, is based on the WBS hierarchy, and is resourceloaded before the construction/implementation stage.
Baseline schedule does not include schedule contingency.

4. Construction
Project Definition

4.10 Schedule
Contingency

Schedule contingency amounts and project end date with
contingency; state method of calculating contingency,
including confidence level for meeting project end date.

5. Staffing

5.1 Staffing Plan

Staffing FTE plan, per NSF and other project-specific job
categories, over time.

5. Staffing

5.2 Hiring and Staff
Transition Plan

Schedule and requirements for hiring and training staff,
including timelines for increasing or decreasing staffing
levels. Required qualifications for key staff.

6. Risk and
Opportunity Mgt

6.1 Risk Management
Plan

Risk Management Plan describes the methodology/process
for identifying, ranking, analyzing, tracking, controlling, and
mitigating risks.

6. Risk and
Opportunity Mgt

6.2 Risk Register

A tracking document or tool that provides a ranked list of
identified risks, with risk impact analysis and prioritization,
responsibilities, mitigation plans and opportunities of risk
reduction, and risk status over time.

6. Risk and
Opportunity Mgt

6.3 Contingency
Management Plan

Contingency management plans and approval process using
change control. Describe NSF approval requirements per
cooperative agreements (CAs).

7. Systems
Engineering

7.1 Systems
Engineering Plan

Systems Engineering Management Plan; roles and
responsibilities.

7. Systems
Engineering

7.2 Systems
Engineering
Requirements

System-level design and technical feasibility study, including
definition of all functional requirements and major systems.

7. Systems
Engineering

7.3 Interface
Management Plan

Identification of interfaces between major components or
WBS elements and plans for managing communication,
interferences, and interactions. Interface Management Plan
and Documentation.

7. Systems
Engineering

7. 4 QA/QC Plans

Quality assurance and quality control requirements and
description of processes.

8. Configuration
Control

8.1 Configuration
Control Plan

Configuration Control plans.

8. Configuration
Control

8.2 Change Control Plan

Change Control Plan to manage accounting changes and
changes in the baseline plan: changes in scope,
modifications to budget or schedule, and movement of
contingencies into or out of the baseline. Includes approval
and documentation processes plus roles and responsibilities.

8. Configuration
Control

8.3 Document Control
Plan

Document Control Plan for managing version control, access,
and archiving of project related documentation.

Section Revision:
May 2, 2016

3.4.1-4

Large Facilities Manual: NSF 16-0XX (May 2016)
3.4.1 Components of a Construction-ready Project Execution Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Component

Sub-Topics

Description of Sub-Section Requirements

9. Acquisitions

9.1 Acquisition Plans

Describe acquisition plans, processes, sub-awards, and
subcontracting strategy. Provide a time based list of
acquisitions and procurement actions.

9. Acquisitions

9.2 Acquisition
Approval Process

Describe the approval process for acquisitions (NSF,
internal), and create a year by year Acquisition Plan of
actions that are estimated to require NSF approval.

10. Project Mgt.
Controls

10.1 Project Management Control Plan

Description of the project management organization and
processes.

10. Project Mgt.
Controls

10.2 Earned Value
Management System
(EVMS) Plan

Description of the EVMS plans, processes, software, and
tools.

10. Project Mgt.
Controls

10.3 Financial and
Business Controls

Description of Financial and Business processes and controls.

11. Site and
Environment

11.1 Site Selection

Site selection criteria and description of selected site(s).

11. Site and
Environment

11.2 Environmental
Aspects

List need for any Environmental Impact Statements,
permitting, site assessments, etc.

12. CyberInfrastructure

12.1 Cyber-Security
Plan

Plan for maintaining security of data, hardware, and
networks during all stages of project life cycle.

12. CyberInfrastructure

12.2 Code Development
Plan

Plans for writing, testing and verifying, deploying, and
documenting software, including configuration control
during the stages of development.

12. CyberInfrastructure

12.3 Data Management
Plan

Plans for managing data, including infrastructure, archiving,
open data access plans, etc.

13. Health and
Safety

13.1 Health and Safety
Plans

Safety and Health plans.

14. Review and
Reporting

14.1 Reporting
Requirements

Statement of reporting requirements, including notifications
for specific events and periodic reports on progress and
project technical and financial status per NSF requirements
or CAs.

14. Review and
Reporting

14.2 Audits and
Reviews

Statement of the required and proposed reviews, audits,
and assessments for progressing during project life cycle
through project close-out.

15. Integration
and
Commissioning

15.1 Integration and
Commissioning Plan

Plans for systems integration, testing, and commissioning.

15. Integration
and
Commissioning

15.2 Acceptance /
Operational Readiness
Plan

Plan for operational readiness, including acceptance criteria
and acceptance procedures.

Section Revision:
May 2, 2016

3.4.1-5

Large Facilities Manual: NSF 16-0XX (May 2016)
3.4.1 Components of a Construction-ready Project Execution Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Component

Sub-Topics

Description of Sub-Section Requirements

16. Project
Close-out

16.1 Close-out Plan

Procedures and criteria for closing out the project. Includes
acceptance of verification of technical performance as well
as documented completion of all scope contained in the
WBS dictionary. Includes procedures documentation for
closing out all acquisitions and financial accounting.

16. Project CloseOut

16.2 Transition to
Operations Plan

Plans for transitioning to operational status.

16. Project CloseOut

16.3 Concept of
Operations Plan

Estimate of annual operations and maintenance staffing and
funding that will be needed when the facility is constructed
and operated.

17. Facility
Divestment

17.1 Facility Divestment
Plan

Description and estimate of divestment liabilities at the end
of facility life for transfer, demolition, site remediation,
decontamination, etc., where appropriate.

Section Revision:
May 2, 2016

3.4.1-6

Large Facilities Manual: NSF 16-0XX (May 2016)
3.4.2 Detailed Guidelines for Project Execution Plans
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

3.4.2 Detailed Guidelines for Project Execution Plans
Please contact the cognizant NSF program officer for details, which are given in the internal NSF
draft document, Guidelines for Development of Project Execution Plans for Large Facilities.
This document provides an overview of NSF’s expectations about Project Execution Plans (PEP)
for Program Officers (POs), Grants and Agreements Officers (G/AOs) and others involved in
overseeing a large facility project and assessing the project management plans of a Recipient.
These plans are usually provided, at least in preliminary fashion, as part of the proposal for
construction of a large project. This plan can be fine-tuned during the period following approval
of the award and prior to undertaking construction activities, through interactions between the
Recipient, the NSF PO, and the G/AO that define NSF’s expectations.

Section Revision:
June 17, 2015

3.4.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
3.5.1 Preparation of Proposals for Operations and Maintenance
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

3.5

OPERATIONS PLAN

3.5.1 Preparation of Proposals for Operations and Maintenance
In order to avoid funding gaps, formal proposals to operate a facility should be prepared well in
advance of the anticipated start date for operations: as much as two years prior to the end of
construction and commissioning activities. Program Officers (PO) and Directorates/Offices are
encouraged to take into account the time needed for internal NSF review, including NSB review,
and offer guidance to the community. Estimates of the funds for operations and maintenance
are provided even in the planning stages of a facility. The potential Recipient and/or the PO
need to establish a dialogue with the user community to determine the resources needed to
fully exploit the facility. In addition, the proposal should include:





All costs to operate, maintain and periodically upgrade the facility, its instrumentation
and the IT components, including cost and approximate time of investment (Note: A PO
can expect that IT components will need to be upgraded at least every 3 to 5 years);
The costs of an in-house research program (as a separate line item in the budget), if
applicable, including an indication of how the overall research program will be managed
and how research program resources will be allocated;
Education and outreach plans and costs;
A detailed management plan for operations of the facility, including the roles of key
staff and plans for advisory committees.

Note that cost estimating methods should follow the Government Accountability Office (GAO)
Cost Estimating Guidelines, per Section 4.2.
The review of the proposal includes a realistic assessment of the costs to operate and maintain
the facility in a safe and effective manner. The PO is also responsible for oversight of
operational facilities through the various reviews and reports described in the Internal
Management Plan (IMP). In addition to following the procedures referenced as appropriate to
Chapters V and VI of the Proposal and Award Manual (PAM), the PO considers (with the
assistance of external reviewers with expertise in managing comparably scaled facilities) these
questions:





Is the facility ready for reliable operations and is the infrastructure (including personnel
requirements) adequate to execute the proposed work plan?
Do the operations and maintenance plans allow for optimal utilization of the facility by
users (e.g., scheduled operating time versus down-time)?
Is there an appropriate balance between in-house research and research of external
users?
Are safety (including IT security and security of the physical plant), environmental and
health issues, if any, addressed?

Section Revision:
June 17, 2015

3.5.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
3.5.1 Preparation of Proposals for Operations and Maintenance
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)





Are plans for securing human subjects and/or vertebrate animal clearances included, if
applicable (e.g., assessments of education-related activities)?
Are the Educational Outreach and Broader Societal Impact plan and cost reasonable and
include an appropriate strategy to evaluate the outcomes?
Have all costs been considered and estimated and is the available funding sufficient, or
is some adjustment needed?

Throughout the operational stage, the Recipient operates and maintains the facility in
accordance with the terms and conditions outlined in the CA. The PO, together with the
Division of Grants and Agreements (DGA) or the Division of Acquisition and Cooperative
Support (DACS), drafts the CA that will govern the operational phase of the project in
accordance with the procedures contained in Chapter VIII of the PAM. The CA will include plans
for NSF oversight, reflect the needs of the facility users, and address how the user program will
be managed and how user time will be allocated. The PO provides oversight for all aspects of
operations, maintenance and the research and education program. The PO also maintains an
awareness of emerging technical, managerial and financial issues through contact with the
facility managers and users, and through oversight, reviews and reports.

Section Revision:
June 17, 2015

3.5.1-2

Large Facilities Manual: NSF 16-0XX (May 2016)
3.5.2 Procedures for Renewal or Recompetition of an Operating Large Facility
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

3.5.2 Procedures for Renewal or Recompetition of an Operating Large Facility
At least two years prior to the expiration of an award for operations of a facility, 1 the Program
Officer (PO) will plan a review of the results of research and education, the affected
community’s needs, and the facility’s management, including the performance of its managing
organization. The reviews will be used to determine whether to renew the award, upgrade the
facility, recompete the award or terminate operation of the facility. If the reviews show that the
facility is of low priority relative to other funding opportunities within the field(s) of research
served by the facility, or is otherwise not meeting its goals and objectives, the PO, working with
the Division Director (DD) and Assistant Director (AD)/Office Head, will prepare a plan for either
upgrading the facility’s capabilities or divesting/terminating support.
The review should analyze the costs and benefits of the facility, taking into consideration the
following issues:








How much does the community need the facility, and is the community strong and
actively engaged in utilizing it?
Is the facility meeting the research, educational outreach and broader societal Impact
goals and objectives originally proposed?
Has the facility reached its annual performance goals, and if not, what are the reasons
for not meeting any goals?
Will meeting the goals and objectives place the United States in a leading position
within the research areas served by the facility?
Is the facility a high priority of the field, as established by long-range planning?
Is the facility operating in an efficient and cost-effective manner, or are there
alternative, more efficient and cost-effective ways to meet the need?
What research opportunities and education opportunities elsewhere are being lost by
continued support of this facility?

Federally Funded Research and Development Centers (FFRDCs) follow a slightly different
process and cannot be renewed or divested until a comprehensive review is performed. The
review should meet the requirements outlined in the Federal Acquisition Regulations (FAR Part
35.017-4, Reviewing FFRDCs): An FFRDC review should include the following: (1) an examination
of the sponsor's special technical needs and mission requirements that are performed by the
FFRDC to determine if and at what level they continue to exist; (2) consideration of alternative
sources to meet the sponsor's needs; (3) an assessment of the efficiency and effectiveness of
the FFRDC in meeting the sponsor's needs, including the FFRDC's ability to maintain its
objectivity, independence, quick response capability, currency in its field(s) of expertise, and
familiarity with the needs of its sponsor; (4) an assessment of the adequacy of the FFRDC
management in ensuring a cost-effective operation; and (5) a determination that the criteria for
1

The PO should exercise judgment and consider the complexity of the facility in determining whether to begin the review
process earlier.

Section Revision:
May 2, 2016

3.5.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
3.5.2 Procedures for Renewal or Recompetition of an Operating Large Facility
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

establishing the FFRDC continue to be satisfied and that the sponsoring agreement is in
compliance with FAR 35.017-1.
If the reviews show that the facility remains a high priority and has been successful in meeting
its goals and objectives, the Originating Organization considers whether renewal of the
operating agreement with the Recipient institution, or recompetition, is in the best interests of
NSF and the affected community. In deciding whether to renew or recompete, the PO will take
into consideration that the NSB has expressed its preference for recompeting all awards
periodically. Awards may be renewed without recompetition or with only limited competition if
there is sufficient justification (e.g., facilities or facility sites with special features that preclude
relocation or recompetition, or partnership-related complexities that prevent recompetition).
After the appropriate review has been completed, the PO analyzes what can and what needs to
be done in light of the available funding, and recommends one of the following actions:







Recompete the award;
Renew NSF support;
Renew NSF support and plan upgrades to the facility;
Renew NSF support to allow operations to transition to self-sufficiency (through, for
example, institutional, industrial or other modes of support);
Renew NSF support to allow operations to ramp-down, leading to divestment; or
Terminate NSF support and divest.

In the event that a decision is made to recompete or to terminate support for a facility, the PO
will give the incumbent Recipient as much notice as possible, but not less than one year, so that
all necessary arrangements to transfer (in the case of unsuccessful recompetition by the
incumbent management entity) or terminate obligations to vendors and employees can be
planned and implemented.
In most cases of recompetition, the managing organization of a facility is required to compete
with other organizations for continuation of the management of the facility and renewal
proposals are received from the Recipient institution and/or from other institutions. The
proposal(s) is (are) merit reviewed in accordance with procedures in Chapters V and VI of the
PAM. The normal thresholds for Director’s Review Board (DRB) and NSB award approval apply.1

1

Refer to the footnotes in Section 2.1.6 and 2.7.1 on award thresholds requiring DRB and NSB approval.

Section Revision:
May 2, 2016

3.5.2-2

Large Facilities Manual: NSF 16-0XX (May 2016)
3.5.3 Detailed Guidelines for Oversight of Operations
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

3.5.3 Detailed Guidelines for Oversight of Operations
Please contact the cognizant NSF program officer for details, which are given in the internal NSF
draft document Guidelines for Operations.
This document provides guidelines conducting oversight of the operational phase of NSF’s large
facilities. It elaborates on the principles outlined in the large facilities manual, and offers
additional information and examples that should be especially helpful to individuals newly
involved in operational oversight.

Section Revision:
June 17, 2015

3.5.3-1

Large Facilities Manual: NSF 16-0XX (May 2016)
3.6 Facility Divestment Plan
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

3.6

FACILITY DIVESTMENT PLAN

(Intentionally left blank – this section to be written)

Section Revision:
May 2, 2016

3.6-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.1 Introduction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

4
4.1

KEY MANAGEMENT PRINCIPLES AND REQUIREMENTS FOR LARGE FACILITIES
INTRODUCTION

This section provides greater detail about key management, budgeting, and reporting activities
that should be carried out throughout a project’s life cycle stages, for both Major Research
Equipment and Facilities Construction (MREFC) and non-MREFC projects, to ensure adherence
to principles established by National Science Foundation (NSF).
Some of these activities will be funded via MREFC and others via R&RA, depending on lifecycle
stage.

Section Revision:
May 2, 2016

4.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.1 Cost Estimating and Analysis Overview
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

4.2

COST ESTIMATING AND ANALYSIS

4.2.1 Cost Estimating and Analysis Overview
This guidance clarifies NSF expectations for the format, content, supporting justification, and
best practices for Recipient cost estimates. This guidance also explains the NSF cost analysis
process and timeline. By following this guidance Recipients should expect a better estimate and
a more efficient review by NSF, facilitating achievement of the science mission. Recipients
should note any departures from this guidance and explain their rationale. For existing awards,
the Recipient should consult with the PO.
NSF uses internal staff, outside experts, and panel reviews to analyze estimates and ensure
construction and operations awards are allowable, allocable, reasonable, and realistic.
Allowable costs are defined by federal guidelines and relevant cost principles. Allocable costs
must be logically related to the particular award. Reasonable costs are what a prudent
individual would pay in a competitive marketplace (i.e., costs are not too high). Cost realism
defines whether the costs are realistic for the work to be performed, reflect a clear
understanding of the requirements, and are consistent with the methods of performance and
materials described in the Recipient’s technical proposal (i.e., costs are not too low).
As described in Section 2 of this manual, Recipients must develop estimates for design,
construction, operation and decommissioning of facilities. Estimates should be well
documented, comprehensive, accurate, and credible and should facilitate appropriate analyses
from a wide variety of reviewers at the various life cycle stages. It is understood that cost
estimates will undergo further refinement at each stage-gate review and the materials required
herein will evolve accordingly. NSF will review estimates at an appropriate level as the project
advances through the design stage.
The NSF PO, G/AO, and Cost Analyst conduct a detailed analysis of the Recipient cost estimate.
NSF also utilizes external panel reviews and independent cost estimate reviews to inform the
analysis. The G/AO and Cost Analyst review includes the detailed sub-elements, cost categories,
and supporting basis of estimate discussed in this section of the manual. The PO review
includes the technical scope, risk, level of effort, and schedule assumptions. The LFO supports
analysis of the proposed contingency budget. The inputs from the various sources are
integrated and reconciled with the Recipient estimate. The PO ultimately recommends the
budget, funding profile, and internal and external sources of funds based on the results of the
cost analysis and the availability of funds.
Figure 4.2.1-1 and Figure 4.2.1-2 below depict when cost analyses are performed during the
design and operations stages. NSF may also perform cost analyses at other times, as necessary,
based on a risk-based assessment. For example, cost analyses may be needed to support
significant changes in scope, schedule, or cost (e.g., for large dollar amounts or relative percent
increases in expected costs), or when upcoming annual project plans include items that are

Section Revision:
May 2, 2016

4.2.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.1 Cost Estimating and Analysis Overview
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

considerably risky or complex. These latter types of analysis may only require review of
targeted subset of information for specific changes.
NSF typically requires 90 to 180 calendar days to complete a full review and detailed cost
analysis of a proposal budget prior to proceeding to the next design phase or prior to award for
operations or construction. This time will vary depending on project scope, cost, risk,
complexity, and relative importance. During this time, 60 calendar days should be allocated for
the cost analysis by the CAAR Cost Analyst.
If the Recipient provides insufficient or conflicting information, the G/AO or CO and Cost
Analysts may require additional documentation and justification and further interaction with
the Recipient prior to making a recommendation. Communication among all parties as well as a
sound initial basis of estimate are essential for timely and successful completion.
When submitting packages for cost analysis, Recipients must submit the following as a
minimum:






Cost Estimating Plan per Section 4.2.2.1.
“Cost Model Data Set” per Section 4.2.2.1.
Reports and Proposals per Sections 4.2.2.2 and either 4.2.3.2 or 4.2.4.2.
The Work Breakdown Structure (WBS) and WBS Dictionary per either Sections 4.2.3.3 or
4.2.4.3.
Supporting information forming the Basis of Estimate (BOE) per Sections 4.2.2.4, 4.2.2.5,
4.2.2.6, and either 4.2.3.4 or 4.2.4.4

Section Revision:
May 2, 2016

4.2.1-2

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.1 Cost Estimating and Analysis Overview
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative Support (BFA-DACS), & The Large Facilities
Office (BFA-LFO)
Figure 4.2.1-1

Section Revision:
May 2, 2016

Cost Analysis Process during Design Phases for Construction Awards (Post – CDR, PDR, FDR to Support MREFC Stage-Gate Review
Process)

4.2.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.1 Cost Estimating and Analysis Overview
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative Support (BFA-DACS), & The Large Facilities
Office (BFA-LFO)
Figure 4.2.1-2

Section Revision:
May 2, 2016

Cost Analysis Process for Operations Awards

4.2.1-2

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.2 Cost Estimating and Analysis Process
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

4.2.2 Cost Estimating and Analysis Process
4.2.2.1 Cost Estimating Plan
For new awards, Recipients must develop and submit a Cost Estimating Plan (CEP) to clearly
articulate cost estimate assumptions in order to support the cost analysis. NSF will review the
CEP to ensure the Recipient plans are aligned with NSF expectations outlined herein. The CEP
should address the general guidance in Section 4.2.1 and the specific guidance from either
Section 4.2.3 or 4.2.4 for construction projects or operational awards, respectively. For
construction awards, the CEP must explain how the Recipient will follow “The Twelve Steps of a
High-Quality Cost Estimating Process” from the GAO Cost Estimating and Assessment Guide:
Best Practices for Developing and Managing Capital Program Costs (GAO-09-3SP March 2009, or
subsequent revision). For existing awards, the Recipient should consult with the PO regarding
the CEP.
In general, the CEP should describe how the guidance in Section 4.2 of this manual will be
implemented, how the cost estimate will evolve over time, and how the “Cost Model Data Set”
will meet the various needs of the project. The “Cost Model Data Set” is the cost data used as
input to software tools and/or project reports to organize, correlate, and calculate different
project management information. The CEP should also typically include a narrative and
sufficient detail explaining the ground rules and assumptions, practices, systems, and
calculations used to develop the cost estimate. The CEP should describe the expected cost
estimating methodology at each stage (e.g., expert opinion, analogy, parametric, or engineering
build-up). Recipients should also discuss their own independent estimates and reviews, if any,
planned to validate the project estimate.
Figure 4.2.2-1 provides an example of a how a “Cost Model Data Set,” Work Breakdown
Structure, and a Recipient’s institutional accounting systems can be used as inputs in
conjunction with scheduling, earned value, and risk analysis tools to generate a variety of
output reports for project management purposes. Sections 4.2.3 and 4.2.4 of this manual refer
specifically to the “Cost Model Data Set” and “Cost Reports” blocks encircled with dashed lines
in Figure 4.2.2-1. The CEP is included as part of section 4.7 of the PEP as described in Section 3.4
of this manual.
The CEP should be tailored to address all relevant stages and costs of the facility lifecycle, from
Development and Design through Construction, Operation, and Termination. For example, the
expected level of funding needed for the Operations Stage should be identified at the
Conceptual Design Review. Operating costs estimates will be updated throughout the design
and construction process as further discussed in the Concept of Operations Plan developed as
part of the PEP described in Section 3.4 of this manual.

Section Revision:
May 2, 2016

4.2.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.2 Cost Estimating and Analysis Process
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative Support (BFA-DACS), & The Large Facilities Office
(BFA-LFO)
Figure 4.2.2-1

Section Revision:
May 2, 2016

Sample Project Management Control Systems Flow Chart

2

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.2 Cost Estimating and Analysis Process
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

4.2.2.2 Relevant Guidance and Reporting Formats
Proposed budgets must comply with the applicable federal regulations, as implemented by NSF
in the Proposal and Award Policies and Procedures Guide (PAPPG), including Grant Proposal
Guide (GPG) and Award and Administration Guide (AAG). Recipients are required to follow the
best practices within the GAO Cost Estimating and Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs (GAO-09-3SP March 2009, or subsequent
revision) and GAO Schedule Assessment Guide: Best Practices for Project Schedules (GAO-1689G December 2015, or subsequent revision), taking into consideration NSF policy and practice
as provided in this manual. Recipients must note any departures from these GAO guides and
explain their rationale. The Recipient should also be prepared for a review of all of its business
systems as described in Section 4.5.3.3 in accordance with NSF’s Business Systems Review (BSR)
Guide.
The “Cost Model Data Set” must be capable of providing cost information in multiple formats
and reports, including but not limited to the following:




Reports based on a deliverable-based work breakdown structure (WBS) for construction
and a functional activity and/or deliverable based work breakdown structure for
operations, as further described in Sections 4.2.3.3 and 4.2.4.3 below. These reports
support project management and execution and detailed cost analysis of sub-elements
and are referred to as “Cost Book Reports.”
In the standard NSF budget format, depicted in Figure 4.2.2-2 and Section 4.2.2.4 below,
per NSF budget and budget justification guidance from the GPG in the PAPPG. This
format supports cost analysis of broad NSF budget categories.

The estimate is built-up from the individual WBS elements and sub-elements. If the costs
associated with each WBS element are binned into the appropriate NSF budget categories, then
both of the above reporting formats can be readily produced. For example, costs can be coded
with NSF budget format letters (A through I per Figure 4.2.2-2) in the “Cost Model Data Set” to
populate rolled-up NSF budget format summaries as well as “Cost Book Reports” organized by
WBS. Actual expenditures can then be later compared to budgeted amounts using earned value
management (EVM) during construction.
The cognizant NSF PO or G/AO can be contacted with questions or for other specific
programmatic requirements.

Section Revision:
May 2, 2016

4.2.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.2 Cost Estimating and Analysis Process
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Figure 4.2.2-2

NSF Budget Categories Sample Format

A – Senior Personnel
B – Other Personnel
B.1 – Postdoctoral Scholars
B.2 – Other Professionals (Technicians, Programmers, Etc.)
B.3 – Graduate Students
B.4 – Undergraduate Students
B.5 – Secretarial – Clerical
B.6 – Other
C – Fringe Benefits
D – Equipment
E – Travel
E.1 – Domestic
E.2 – Foreign
F – Participant Support
F.1 – Stipends
F.2 – Travel
F.3 – Subsistence
F.4 – Other
G – Other Direct Costs
G.1 – Materials and Supplies
G.2 – Publication, Documentation, Dissemination
G.3 – Consultant Services
G.4 – Computer Services
G.5 – Sub-awards
G.6 – Other
H – Total Direct Costs
I – Indirect Costs

4.2.2.3 Funding Sources and Award Instruments
For most large facilities projects, funding is derived from the appropriate NSF budget account
depending on the Stage: typically the Research and Related Activities (R&RA), Education and
Human Resources (EHR) or MREFC account. Barring documented exceptions, the R&RA (and
possibly EHR) account will be used to fund the Development, Design, Operations, and
Termination Stage costs. The MREFC account will be used to fund construction, acquisition and
commissioning costs as part of the Construction Stage.

Section Revision:
May 2, 2016

4.2.2-2

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.2 Cost Estimating and Analysis Process
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

For non-MREFC projects, the R&RA (and possibly EHR) account also funds the construction,
acquisition and commissioning costs, as discussed in Section 2.7 of this manual.
All lifecycle Stages are funded through the appropriate award instrument (typically a
Cooperative Agreement (CA)) between NSF and the managing organizations (i.e., the
Recipient). Infrastructure utilization is typically supported through grants (R&RA or EHR) funded
by NSF and/or other agencies to conduct research and education activities using the facility.
For projects to be funded through assistance awards (CAs or grants), the PO recommends
award or decline in accordance with the proposal processing procedures contained in Chapter
VI of the Proposal and Award Manual (PAM). The PO, together with staff from NSF’s Division of
Acquisition and Cooperative Support (DACS) and LFO, drafts the award instrument that will
govern the project in accordance with the procedures contained in Chapter VIII of the PAM. For
projects funded through contracts, the Federal Acquisition Regulations apply. Before proposals
for large facilities projects can be awarded, funds for the project must be approved by NSF at
the appropriate level and appropriated by Congress.
Proposed budgets shall also describe any contributions from other agency or international
partners. For awards involving property, the PO consults with the OGC, the Division of Financial
Management, the Property Office, and the LFO Liaison to determine whether the value of the
property should be included on NSF’s financial statements. The LFO Liaison coordinates help
with other BFA management divisions as required.
4.2.2.4 Supplementary Guidance for Construction and Operations Awards
This section discusses additional detailed information needed for cost analysis of broad NSF
budget categories to determine if Recipient proposals are allowable, allocable, reasonable, and
realistic. This information is intended to supplement the standard grant guidance for the NSF
budget categories described in the GPG in the PAPPG as depicted in Figure 4.2.2-2 (NSF Budget
Categories Sample Format). This guidance is intended to clarify NSF expectations, assist
Recipients, facilitate NSF review with fewer iterative resubmissions, and prevent recurrent
issues. Additional guidance on the importance of justifying individual work breakdown structure
elements is provided in Sections 4.2.3 and 4.2.4. Along with the CEP, Recipients should provide
the following supplementary information to support their BOE:
Documentation and Cost Models
Supporting source documentation should be provided, such as current payroll registers, vendor
quotes for items of equipment where available, historical costs, and scaling factor justifications.
The “Cost Model Data Set” should allow for mathematical checks of the proposal budget
calculations and should contain formulas and allow manipulations to check calculations (i.e.,
the model should not be locked).

Section Revision:
May 2, 2016

4.2.2-3

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.2 Cost Estimating and Analysis Process
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

For all salary data provided, Personally Identifiable Information should be removed from the
documentation described below. If not already covered in the CEP, Recipients should provide a
salary escalation rate for multi-year proposals, which can include a component for annual raises
similar to Cost of Living Adjustments (COLAs) and other pay increases for promotions within the
position classification. Recipients should provide the rationale behind the salary escalation rate.
In some cases, NSF may provide a base escalation rate in the solicitation for guidance, but
Recipients should follow Section 4.2.2.6 of this manual when proposing rates.
A – Senior Personnel




Recipients should provide verification of actual salaries paid for named senior
personnel. Salary rates should be based on actual costs per current rate paid by payroll
register, W-2s, or appointment letters. Recipients should note Academic Year (9-10
month) versus Calendar Year (12 month) appointments or time available to conduct
independent research if such appointments so provide. The Recipient should also
provide documentation to support reasonableness of the salary rate paid, such as salary
rate surveys, salary comparators, Human Resource Department analysis, or other
information.
NSF has a policy which typically limits senior personnel to two months effort in any
given year for standard NSF-funded grants. However, most grants and cooperative
agreements for large facility projects (both construction and operations) have senior
personnel effort well in excess of two months. Compensation in excess of two months, if
anticipated, should be disclosed in the proposal budget, and explained in the budget
justification.

B.1 – Postdoctoral Scholars


Recipients should provide the average salary rate or rate range for postdoctoral
students at the organization in the field of science. Actual payroll data may not be
available as these may be To-Be-Hired (TBH) positions.

B.2 – Other Professionals, Technicians, Programmer, Etc.)


This category is of special concern in both construction and operations proposals for
large facility projects. The level of effort will likely need to be obtained by individual or
by position for salary calculations. Since the NSF budget format poses this as a total
number of individuals for a total number of months, additional explanation is generally
required to disaggregate the total for cost analysis. Recipients should also provide a
spreadsheet with the budget justification that includes: name or position number,
location, activity or function (operations awards see Section 4.2.4.3) or WBS
(construction awards see 4.2.3.3), title, salary rate and period, level of effort as a
percentage or in person-months, and calculation of amount for each award year.

Section Revision:
May 2, 2016

4.2.2-4

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.2 Cost Estimating and Analysis Process
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)



Recipients should provide supporting documentation for the salary rates of the
technicians, programmers, and other professionals proposed. For these types of
positions, NSF recommends the use of Bureau of Labor Statistics (BLS) Standard
Occupation Classification Codes (SOC) by position title and referencing their positions to
BLS salary rates to establish reasonableness of proposed salary rates. The BLS data is
also available by region or city. Other salary rate survey data may be used and larger
Recipient organizations may already have established salary ranges and qualification
bases established internally by their Human Resources Departments.

B.3 – Graduate Students


Recipients should provide the average salary rate or rate range for graduate students at the
organization in the field of science. Actual payroll data may not be available as these may be
TBH positions.

B.4 – Undergraduate Students


Recipients should provide the average salary rate or rate range for undergraduate students at
the organization in the field of science. Actual payroll data may not be available as these may
be TBH positions.

B.5 – Secretarial – Clerical


Recipients should provide the average salary rate or rate range for secretarial clerical personnel
at the organization.

B.6 – Other Personnel


Generally the same as B.2 above but special classifications could justify different treatment.

C – Fringe Benefits




Most Recipient organizations utilize a single tier fringe benefit rate or fringe benefit rate by class
of employee. Occasionally these fringe benefit rates are approved in the Negotiated Indirect
Cost Rate Agreement (NICRA). In such cases, the Recipient can verify the rate and provide a
fringe benefit calculation (rates by class) for a sample project year. These cases should be noted
in the CEP.
Some organizations use an actual fringe benefit amount by class of employee. These amounts
vary greatly by employee salary levels. While some fringe benefit costs are based on a
percentage of salaries (such as statutory withholding or contributions to retirement and Paid
Time Off (PTO)), other fringe benefits such as medical insurance may be a lump sum amount and
are not directly tied to salary paid. The Recipient should provide an estimate of each fringe
benefit provided as a percentage to salaries paid along with a description of the fringe benefit
provided as a means to gauge the reasonableness of the fringe benefit package provided.

Section Revision:
May 2, 2016

4.2.2-5

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.2 Cost Estimating and Analysis Process
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)



In both cases, Recipients should explain differences in the treatment of PTO. Some organizations
include this as a component of the fringe benefit rate and others include the full cost of salary
(including PTO) in the salaries as budgeted.

D – Equipment


Equipment is defined as items with an acquisition cost and a useful life above established
thresholds. The acquisition cost includes modifications, attachments and accessories necessary
to make the property useful for the purpose for which it was purchased. There can be
equipment expenses or materials and supplies that individually are less than the threshold but
taken together exceed the equipment threshold, particularly when installed or fabricated by a
Recipient.



Recipients should list each item of equipment individually and include a description,
estimated cost, and justification of need. Recipients should provide vendor or catalogue
quotes for each item of equipment. These quotes should be indexed and numbered to
the equipment items proposed. For items of equipment exceeding the small purchase
threshold, three or more vender quotes or sole source justification should also be
provided. For unique scientific instrumentation or other equipment components where
vendor quotes are not readily available, a clear basis of estimate should be provided.
General purpose equipment is generally not supported by NSF funding unless used
primarily on the actual conduct of the NSF-funded project.



E – Travel


Travel activities should be documented by individual’s destination and cost. Generally the cost
should be broken out by transportation (airfare or mileage), per diem (lodging and meals) and
other associated expenses. The relation of the travel to the proposed activities should also be
included. For renewal projects, historical costs can be considered as a means of assessing the
reasonableness of travel costs. Where there are large numbers of trips and the actual locations
may not be known in advance, then cost estimating relationships (e.g., average of $1,500 per
traveler per trip) may be used.

F – Participant Support




This category includes stipends, travel, subsistence, and other costs for participants in an NSF
sponsored conference, workshop or other training activities. Justification should include the
number of participants, stipend amount, travel cost estimate, and subsistence costs per
participant. Recipients should also provide the number of days or weeks of the training
activities to provide a basis for determining reasonableness of the proposed payments.
Participant support costs may not be used for personnel at the Recipient institution.

G.1 – Materials and Supplies


The general types of expendable materials and supplies should be indicated. An itemized listing
is not necessary unless an item represents a substantial amount of costs. Vendor or catalogue

Section Revision:
May 2, 2016

4.2.2-6

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.2 Cost Estimating and Analysis Process
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

quotes, historical costs, or other cost estimating relationships may be used to establish
reasonableness of the cost estimate.

G.2 – Publication, Documentation, Dissemination


Recipients should provide an estimate of publication and dissemination costs.

G.3 – Consultant Services


For each consultant identified, the Recipient should provide the individual’s expertise, primary
organizational affiliation, normal daily compensation rate, number of days of expected service,
and justification that the proposed rate of pay is reasonable. Consultant travel and subsistence
costs may also be included.

G.4 – Computer Services


Where it is established institutional policy to direct charge computer services, the Recipient may
justify and include such costs in the budget. Generally such recharges should be based on
established internal institution usage rates. Recipients should provide a statement or policy
document and rates by units of actual usage.

G.5 – Sub-awards (Sub-recipients)



Recipients are expected to conduct a pre-award review of the Sub-awards to include both cost
and price analysis and to identify risk as outlined in the Uniform Guidance, 2 CFR section
#200.331.
Recipients should provide NSF with their pre-award analysis of each of the proposed Subawards. Such Recipient pre-award analysis should include a determination of Sub-award risk.
This should include an assessment of adequacy of accounting system, financial capability, and
ensuring the Sub-recipient is not on any Federal Government “do not pay” listing. The
Recipient should also have performed a price or cost analysis of the Sub-recipient’s proposed
work to ensure the reasonableness of costs. Sole source awards should also be adequately
justified. NSF reviews the Recipients documentation on each Sub-recipient to ensure sufficient
rigor and detail was performed. Sub-recipient analysis should typically be performed by
internal cost analysis and budgeting personnel.

G.6 – Other


Budget contingency should also be presented as a part of the total amount of Other
Direct Costs and included in section G.6 on the standard NSF budget form. Budget
contingency budget estimates should be developed in accordance with Sections 4.2.5
and 5.2 of this manual. Budget contingency and allocations of contingency will be called
out in the Cooperative Support Agreement by the G/AO under the “Contingency”
section, based on information provided in the negotiated budget justification.

Section Revision:
May 2, 2016

4.2.2-7

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.2 Cost Estimating and Analysis Process
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)



Itemized Other costs, including the budget contingency, should be listed separately in
the Comments area of the form.

I – Indirect Costs



When the Recipient has a NICRA established with a cognizant federal agency, the rate and base
in that agreement should be used to compute indirect costs. A copy of the NICRA should be
included in the CEP.
When a Recipient does not have a NICRA, the Recipient should provide a calculation and an
indirect cost rate proposal. The Recipient should ensure that indirect costs are in accordance
with NSF policies in NSF’s Indirect Cost Rate Proposal Submission Procedures. Recipients should
provide a clear description of rates and application bases. Recipients should also provide
spreadsheet calculation of rate or rates by year clearly showing exclusions such as sub-awards
greater than $25,000, equipment or capital expenditures, and participant support. If a Recipient
has different indirect cost rates across NSF budget categories in Figure 4.2.2-2, these rates
should be clearly identified and justified. Any deviation to a Recipient’s normal rate should also
be justified.

4.2.2.5 Management Fees
Management fee is an amount of money paid to a recipient in excess of a cooperative
agreement’s or cooperative support agreement’s allowable costs. Generally, NSF does not
permit the payment of fee (profit) to organizations under financial assistance. However, a
management fee may be authorized for awards in the limited circumstances of construction or
operations of a large facility as the responsible organization is likely to incur certain legitimate
business expenses that may not be reimbursable under the governing cost principles. NSF
provides for a management fee in these limited circumstances, as appropriate, for example
recognizing that the recipient may only incur such expenses as a result of its support of the
NSF-funded activity.
Prior Approval of Management Fees - A management fee proposal must be submitted to NSF
that provides sufficient visibility into each e x p e n s e category to identify its intended
purpose. Agreement on management fee amounts shall be completed and a specific dollar
amount established prior to the initiation of work under an award, or any subsequent period
not authorized as part of the initial award. Any amount negotiated shall be expressly set forth
in the terms and conditions of the award. Recipients may draw down the management fee in
proportion to the amount incurred during the performance period. Fee established for a
period longer than one year shall be subject to adjustment in the event of a significant change
to the budget or work scope.
The following expense categories will be used in the negotiation and award of a management
fee:

Section Revision:
May 2, 2016

4.2.2-8

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.2 Cost Estimating and Analysis Process
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)



Working capital necessary to fund operations under an award - An amount for working
capital may be necessary to ensure a level of retained earnings available to the
organization in order to secure credit and borrowing to assure the financial health of the
organization.



Facilities capital necessary to acquire assets for performance - An amount for facilities
capital may be necessary to allow the organization to acquire major assets and to
address expenses that require immediate substantive financial outlays but that are only
reimbursed through depreciation or amortization over a period of years.



Other ordinary and necessary expenses for business operations that are not otherwise
reimbursable under the governing cost principles – An amount for other expenses that
are ordinary and necessary but not otherwise reimbursable may be appropriate to
provide a reasonable allowance for management initiative and investments that will
directly or indirectly benefit the NSF-funded activity. Inclusion of amounts under this
category warrants careful consideration of the benefits that may be obtained when
providing management fee. Examples of potential appropriate needs include expenses
related to contract terminations and losses, certain appropriate educational and public
outreach activities, and financial incentives to obtain and retain high caliber staff.



Prohibited Use of Management Fees - Although not an exhaustive list, the following are
examples of expenses that are not appropriate uses of a management fee:
o Alcoholic beverages
o Tickets to concerts, sporting and other events
o Vacation or other travel for non-business purposes
o Social or sporting club memberships
o Meals or social activities for non-business purposes
o Meals or social activities for business purposes that are so extravagant as to
constitute entertainment
o Luxury or personal items
o Lobbying as set forth at 2 CFR § 200.450 and FAR 31.205-22, as appropriate to the
recipient type

In addition, costs incurred under the award that are otherwise allowable under the governing
cost principles must be classified as direct or indirect charges to the award and shall not be
included as proposed management fee elements.
Documentation Requirements on Use of Management Fees - Even though the management
fee represents an amount in excess of allowable cost and is therefore not subject to the
governing cost principles, NSF, as a matter of policy, has determined that review of appropriate
use of such funds is necessary. Information available on actual uses of management fee

Section Revision:
May 2, 2016

4.2.2-9

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.2 Cost Estimating and Analysis Process
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

previously awarded by NSF in the preceding five-year period under any award shall be included
in the proposing organization’s fee proposal. As a term and condition of the award, the
recipient will be required to provide information (typically annually) on the actual use(s) of the
management fee. NSF will conduct reviews of this information regarding the extent to which
the recipient fee proposals have proven reliable when compared with actual uses of
management fee (both as to the fee amount as well as the planned uses of the fee).
Unexplained failure to reasonably adhere to planned uses of fee will result in reduction of
future management fee amounts under the award.
4.2.2.6 OMB Inflators
Recipients are not limited to using the publically available economic assumptions and broad
OMB inflators (https://www.whitehouse.gov/omb/budget/Supplemental) when doing cost
estimates. NSF encourages organizations to use inflators appropriate for the known situations
or a particular industry as long as they can be justified. For example, specialized data may be
available from the Department of Energy, Department of Defense, BLS, industry metrics, and/or
historic experience with similar items. The justification for all inflators (including use of
standard OMB inflators) should be included in the CEP and inflators should be used consistently
throughout the BOE in accordance with the CEP.
Estimates should preferably be based on current information, but may include appropriate
escalation calculations and justifications to support the planned execution timeframe.
Escalation for raw materials and equipment in technological projects often runs higher than
broad measures of inflation (e.g., the consumer price index) due to inelasticity in pricing (i.e.,
there are few or no substitutes available in the marketplace and demand remains constant).
Recipients should consider establishing risks and associated contingency for future price
variability and developing mitigating actions such as periodic validation of prices.

Section Revision:
May 2, 2016

4.2.2-10

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.3 Cost Estimating and Analysis for Construction Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

4.2.3 Cost Estimating and Analysis for Construction Awards
4.2.3.1 Purpose and Process
NSF utilizes internal staff, outside experts, and expert panels at the Conceptual Design,
Preliminary Design and Final Design Reviews and during the Construction Stage to assure that
proposed construction cost estimates and budgets meet expectations and are allowable,
allocable, reasonable, and realistic. Cost Estimating Plans and Cost Reports should be updated
as necessary during each of the Phases in preparation for the Reviews. These Cost Reports
include Construction Cost Book Reports, MREFC Panel Reports, and others as depicted on
Figure 4.2.2-1. NSF also generates various Cost Reports such as the Cost Proposal Review
Document (CPRD) for cost analysis as part of its oversight and assurance roles.
The PDR estimate and subsequent NSF analysis must be sufficient to give NSF confidence in the
Not-To-Exceed estimated Total Project Cost (TPC) that advances for National Science Board
approval and potential inclusion in a future budget request. The FDR estimate and analysis
must be sufficient to give NSF confidence in constructing and commissioning the facility within
the TPC.
As denoted in Section 4.2.2.2, Recipients must follow the best practices within the GAO Cost
Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program
Costs (GAO-09-3SP March 2009, or subsequent revision) and GAO Schedule Assessment Guide:
Best Practices for Project Schedules (GAO-16-89G December 2015, or subsequent revision),
taking into consideration NSF policy and practice as provided in this manual. Recipients should
note any departures from these GAO guides and explain their rationale. Some specific GAO best
practices are highlighted below to show how they should be integrated with NSF processes.








Define the estimate’s purpose, develop an estimating plan, and identify ground rules
and assumptions as described further by the CEP in Section 4.2.2.1.
Define project characteristics as described further by the PEP in Section 3.4.
Present the estimate in accordance with the project WBS as described further in
Sections 4.2.3.2 and 4.2.3.3 below.
Obtain data and develop a point estimate. The estimating method, level of detail, and
accuracy range will evolve and improve through the design Phases of the project as
described further in Sections 4.2.3.4 below. Far in advance of stage gate reviews, POs
should work with LFO and DACS to determine the estimating methods appropriate for
each project.
Conduct a sensitivity analysis to examine the effects of changing assumptions and
ground rules so that decision makers know the range of potential costs that surround a
point estimate and the major cost drivers.
Conduct a risk and uncertainty analysis as described further in Sections 4.2.5 and 5.2
below.

Section Revision:
May 2, 2016

4.2.3-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.3 Cost Estimating and Analysis for Construction Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)








Use the “Eight Types of Independent Cost Estimate Reviews” as overarching guidance
for independent cost estimate reviews of large facilities projects. These reviews validate
the estimate and help determine whether the estimate is well documented,
comprehensive, accurate, and credible. Recipients may choose to perform their own
independent cost estimate reviews before submitting proposals to NSF. Far in advance
of stage gate reviews, POs should work with LFO and DACS to determine the type of
review appropriate for each project.
The results of an independent cost estimate review will be factored into the NSF cost
analysis. To ensure maximum usefulness of the review, it will generally be done as part
of the Preliminary Design Phase. This will ensure that analysis is available for feedback
to the Recipient prior to commencing the Final Design Phase and to help establish the
MREFC budget request to the NSF Director, the NSB, OMB, and Congress post-PDR. The
review will be reconciled with the Recipient’s Preliminary Design Phase Cost Estimate in
the CPRD and all findings and conclusions should be substantially resolved prior to any
agreement for construction being awarded.
Document the estimate and present the estimate for review as described further in
Sections 4.2.2.4, 4.2.2.5, and 4.2.2.6 above and 4.2.3.2, 4.2.3.3, and 4.2.3.4 below.
Update the estimate to reflect changes and actual costs including EVM, as support by
Sections 4.2.3.3 and 4.2.3.4 below, and further described in Sections 3.4.1 and 5.7.

4.2.3.2 Construction Cost Reports Overview
Construction Cost Reports are necessary at the CDR, PDR, and FDR, at minimum, to provide a
comprehensive, consolidated estimate of construction costs, including baseline costs and
contingency.
The Project Execution Plan described in Section 3.4 of this manual includes a Construction Cost
Book Report as one component of the overall Construction Project Definition. The Cost
Estimating Plan and Construction Cost Book Reports provide assumptions and detailed
information forming the Basis of Estimate. The following additional high level information
should typically be provided as an overview and executive summary to assist with the review
process described in Section 2 of this manual. Recipients should consult with the PO and G/AO
as necessary to identify any other specific cost reports and content required to support the
review.




An executive summary of the overall project, including a narrative explaining the project
scope (description, location, facility/vessel(s), planned science work, timeline,
stakeholders), the overall basis of estimate, common assumptions across all WBS areas,
and any other significant factors or considerations.
A comparison of the current estimate to the approved cost, schedule and scope
requirements defined in the project baseline, and an explanation and corrective action
for any deviations.

Section Revision:
May 2, 2016

4.2.3-2

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.3 Cost Estimating and Analysis for Construction Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)












Reference to the Scope Contingency portion of the PEP, reinforcing how de-scoping
strategies are possible and realistic without impairing the project’s scientific value.
Reference to the CEP cost inflators and estimate methodology.
The overall WBS and WBS Dictionary should be provided, highlighting any changes (A
sample WBS Format is provided in Figure 4.2.3-1 below).
Overall high level cost summary charts, tables, profiles, and reports; depicting total and
annual costs; reported both by WBS and in NSF budget format; providing both Current
Year and Then Year costs.
Explanation of how project costs by WBS map to the NSF budget format, including
detailed traceability or crosswalk matrix, described further below.
Overall High Level Schedule Phasing for the Project (planning, design, engineering,
contracts, permitting, construction phasing, commissioning, start-up)
Budget contingency estimate amounts, including overall contingency as a percentage of
the performance baseline, rolled up from specifically assigned contingencies from a
deliverable at the WBS level, per Sections 4.2.5 and 5.2 of this manual.
Other reports, as needed, e.g. costs by resource types (subcontract, labor, materials,
travel), cost profiles (total, labor, non-labor, by WBS sub-element), personnel profiles
(Full-time-equivalents by WBS sub-element).
Other Relevant Information, as needed, e.g., description of acquisition strategy, contract
type(s), source(s) of funds.
A statement positively affirming Recipient compliance with applicable federal
regulations and NSF policy, including the LFM and GAO Cost Estimating and Assessment
and GAO Schedule Assessment Guides.

4.2.3.3 Construction Cost Book Report Format and Work Breakdown Structure

As described in the GAO Cost Estimating and Assessment Guide, the WBS is the essential
cornerstone of every project because it defines in detail the work necessary to accomplish a
project’s objectives. The WBS is a deliverable-based and hierarchical framework structure that
provides specific, manageable and schedulable baseline tasks and may be composed of
products, material, equipment, services, data, and support facilities that the project must yield.
The WBS provides a consistent framework for planning, estimating costs, developing schedules,
identifying resources, and determining where risks may occur. The WBS is a valuable
communication tool and provides the means for measuring program status using Earned Value
Management. WBSs are developed at varying levels of detail. Generally, the number of levels
employed should be sufficient to identify and measure progress towards achieving deliverables,
assign responsibility, and enable effective management and reporting. The number of
decomposition levels varies depending on the project’s size and complexity, technical maturity,
organizational constraints, acquisition and construction strategies, and management’s
assessment of need.

Section Revision:
May 2, 2016

4.2.3-3

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.3 Cost Estimating and Analysis for Construction Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

Guidance and examples of common WBS elements can be adapted from GAO, DOE, DOD and
other guidance and tailored for NSF projects(U. S. Department of Energy WBS Handbook dated
August 16, 2012) (DoD Standard Practice WBS for Defense Materiel Items, MIL-STD-881C, dated
3 October 2011). The benefits of developing consistent or similar WBSs across projects within
an organization include:







Consistent, clear, and familiar reporting structures and organizational relationships
Improved efficiency and effectiveness of NSF cost analyses
Better characterization of project schedule, scope, and cost
Ease of judging completeness and reasonableness
Better collection and sharing of data and analysis methods across multiple contractors
and projects
Better cost tracking over time, and identification of major cost drivers and systemic
problems across contractors and projects

Examples of potential components of a WBS, common to many NSF plans for construction of
large facilities, are listed in Figure 4.2.3-1 and further described below. Additions and/or
alterations to this list are likely, due to the unique nature of each specific project.
A basic description of each WBS is as follows:
1.0 Project Administration and Management Office – Include activities related to the
management and administration of the project. This includes quality assurance and
safety, reliability, document control, cost/schedule reporting and control systems, and
configuration management.
2.0 Facility Infrastructure and Civil Construction – Includes the design, procurement,
construction, and integration, of the supporting infrastructure. One example is a
telescope and site construction, consisting of the facility enclosure, dome, and
telescope mount.
3.0 Scientific Equipment and Instrumentation – Includes unique and specialized scientific
equipment. For example, field sensors and gages.
4.0 Computers and Cyber-Infrastructure – Includes hardware and software needed to
operate the system and collect and analyze data.
5.0 Systems Integration, Testing, and Commissioning – Includes the overall systems
infrastructure and personnel needed to integrate other WBS elements to ensure they
work correctly together for testing, commissioning, training, and operations.

Section Revision:
May 2, 2016

4.2.3-4

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.3 Cost Estimating and Analysis for Construction Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Figure 4.2.3-1

Construction WBS and Cost Book Report Sample Format

1.0 Project Administration and Management Office
1.1 Project Management Office
1.2 Site Office
1.3 Science Office
1.4 Education and Public Outreach
1.5 Safety and Environmental Assurance
2.0 Facility Infrastructure and Civil Construction
2.1 Sub-element X
2.2 Sub-element Y
2.3 Sub-element Z
3.0 Scientific Equipment and Instrumentation
3.1 Subcomponent X
3.2 Subcomponent Y
3.3 Subcomponent Z
4.0 Computers and Cyber-Infrastructure
4.1 Data Infrastructure
4.2 Data Products
5.0 Systems Integration, Testing, and Commissioning
5.1 Common Utilities and Support Equipment
5.2 Early System Assembly, Integration, and Testing
5.3 Acceptance Testing
5.4 Training
5.5 Science Verification

4.2.3.4 Construction Cost Book Detail and Supplementary Guidance
This section discusses additional detailed information needed for cost analysis of the WBS
deliverable-based and hierarchical framework to determine if Recipient proposals for
construction awards are allowable, allocable, reasonable, and realistic. This information is
intended to supplement the standard GAO best practices and grant guidance from the GPG in
the PAPPG. The guidance should improve project execution, clarify NSF expectations, assist
Recipients, facilitate NSF review with fewer iterative resubmissions, and prevent recurrent
issues. Recipients should note any departures from this guidance and explain their rationale. It
is understood that this information will become further refined as the Design Stages advance.

Section Revision:
May 2, 2016

4.2.3-5

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.3 Cost Estimating and Analysis for Construction Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

Presentation and Linkages






Individual WBS element costs must have a sound, fully justified and documented, and
sufficiently detailed Basis of Estimate. Figure 4.2.3-2 below provides an example
Construction Cost Book Sheet depicting the format and content typically needed to
consolidate the “Cost Model Data Set” and to provide the appropriate level of detail and
BOE. This sheet includes the following information:
o WBS and activity codes and descriptions, per the WBS Dictionary, to index the
cost estimate to a specific deliverable
o Statement of Work describing the scope
o Estimator Name and Date of Estimate
o Resource Descriptions
o Cost Basis Codes describing the estimate methodology (e.g., expert opinion,
analogy, parametric, or engineering build-up)
o Direct Costs with Units and Hours
o Associated Fringe and Indirect Costs
o NSF Budget Category Code corresponding to the budget categories depicted in
Figure 4.2.2-2 and Section 4.2.2.4 above, to allow mapping between WBS subelements in Construction Cost Book Reports and NSF Budget Categories on NSF
Budget Forms
o Basis of Estimate source data, with breakout of sub-elements, typically including
direct input from technical experts in that area with calculations using material
and labor quantities and unit prices, with clear assumptions and sources
referenced
o Risk and uncertainty identification and determination of parameters used in risk
assessment and contingency estimation
o Contingency estimate amounts and/or percentage of baseline, either by cost
item, activity level, or WBS element (optional, may be determined elsewhere)
Estimates must have clear traceability, including the following, as appropriate, for CDR,
PDR, FDR, and Construction:
o The total estimated cost must correlate to construction drawings and
specifications, material take-offs and bills of material, and estimated schedule
durations.
o Lower levels of the WBS must correctly roll-up to the higher levels, and the
application of rates and factors must be consistent with the cost estimating plan,
basis of estimate, supporting rate agreements, and Recipient accounting
practices.
WBS sub-element costs must be readily mapped to NSF Budget Categories depicted in
Figure 4.2.2-2 and Section 4.2.2.4 above; for example:
o If each cost element on the Figure 4.2.3-2 Cost Book Sheet is assigned an NSF
Budget Category Code (e.g., “A for Senior Personnel,” “E for Travel,” “G4 for
Computer Services,” “I for Indirect Costs”), then the WBS elements can readily
be compiled into standard NSF Budget Forms, which depict total cost types

Section Revision:
May 2, 2016

4.2.3-6

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.3 Cost Estimating and Analysis for Construction Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)



across all WBS elements (e.g., all personnel, equipment, travel, indirect, or
computer services costs across all WBS elements)
o If databases are sufficiently detailed, documented, and traceable, then
automatic sorting and summarizing of costs will be facilitated for various
purposes and for different reporting formats.
Cost estimates may be directly linked to scheduling tools, to allow automatic cost
updates with schedule changes.

BOE Refinement Process











Because of the hierarchical nature of the WBS, it is possible, over time, to refine the
level of detail at which the project scope, schedule, and task-based costs are captured.
Throughout the Design Stage the task and cost fidelity will increase, and eventually,
during the construction of the Project, the plans will be fully detailed. As the project
moves through the phases, detailed engineering build-up estimates using current
quotes and prices should be collected, such that the proportion of estimated costs
based on expert opinion, analogy, or parametric estimates is reduced. As the project
finalizes plans for the start of construction the basis of estimate should include more
vendor contract prices.
Direct labor rates, quantities, and skills mix should be justified, including information
from sub-awards.
If using Consultants and Subcontractors, Recipients should carefully justify substantial
consulting costs, including the type of work performed, quantity of time proposed, and
its cost, as compared to potentially less expensive current employee labor to accomplish
the proposed work.
Cost estimates should include adequate funding for project management, including the
use of appropriate project management tools such as project management control
software and associated staff support.
The MREFC construction cost estimate should typically include commissioning, testing
and acceptance of the facility and transition from construction/acquisition to
operations, including funding for staff to perform these activities and train the
operations personnel. Roles change as a project progresses from construction through
commissioning and eventually to operations; time and staffing requirements need to be
carefully calculated in advance, with clear demarcation between MREFC funded scope
and R&RA or EHR funded operations.
Where partnerships are involved, monetary contributions acquisition and eventual
operations and usage should be timely, sufficient, and well documented in the PEP and
IMP.
Cyber-infrastructure technical requirements and costs (both initial cost and continuing
costs of hardware, software, maintenance, upgrades and operations) should be carefully
considered and periodically validated. Rapid advances in computing may require
upgrades as often as every 3 to 5 years.

Section Revision:
May 2, 2016

4.2.3-7

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.3 Cost Estimating and Analysis for Construction Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative Support (BFA-DACS)
Figure 4.2.3-2

Section Revision:
May 2, 2016

Construction Cost Book Sheet Sample Format

4.2.3-8

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.4 Cost Estimating and Analysis for Operations Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

4.2.4 Cost Estimating and Analysis for Operations Awards
4.2.4.1 Purpose
In addition to the specialized scientific expertise required for operations, award solicitations can
also include expectations for budgeting and business systems and operational or financial
reports. As discussed in the NSF PAPPG, individual solicitations, and Section 2.5, these systems
and reports help ensure the science mission can be met in a cost effective way.
NSF utilizes internal staff, outside experts, and panel reviews to ensure cost estimates and
budgets meet expectations and are allowable, allocable, reasonable, and realistic. It is
incumbent on NSF to plan and budget for effective research and educational use of facilities, as
well as the costs to operate and maintain the facility long term. The PO will generally refer to
the funding profile and cash-flow analysis developed by Program or in collaboration w/an
existing Recipient to make budgeting and funding decisions. If anticipated by NSF and as
discussed in Section 2.6 of this manual, proposals will be requested to address partial or full
termination of the facility during the award period, including property decommissioning and
disposition costs and other costs related to employee separations. It is incumbent upon the
Recipient to ensure their operations proposal is complete, appropriate, and reasonable.
4.2.4.2 Operations Awards Proposals Overview
In addition to the Annual Work Plans, Annual Reports, and Annual Operations Reviews
described in Section 2.5.1 of this manual, the following information may be requested via the
operations and management award solicitation:






Periodic plans that may include an executive summary, narrative overview, strategic and
annual objectives correlated to NSF mission needs, and an annual operating budget
focusing on any significant changes from previous plans. Plans may also include
expected scope, milestones, outcomes and impacts, developments, challenges and
opportunities, as necessary.
A functional activity and/or deliverable based work breakdown structure. A sample
format for operating facilities is provided below in Figure 4.2.4-1, if not otherwise
specified in the Programmatic requirements or the solicitation.
Explanation of how program costs within functional areas are coded or otherwise
related to the NSF Budget Categories depicted in Figure 4.2.2-2 and Section 4.2.2.4
above.
Other reports, such as annual cost by resource types (subcontract, labor, materials,
travel), cost profiles (total, labor, non-labor, by sub-element), and personnel profiles
(Full-time-equivalents by sub-element).

4.2.4.3 Operations Awards Proposals Format
As discussed in Section 4.2.3.3, there are numerous benefits of standardizing the work
breakdown structure across projects within an organization. An example of a hierarchical work
Section Revision:
May 2, 2016

4.2.4-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.4 Cost Estimating and Analysis for Operations Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

breakdown structure for an operations award is provided in Figure 4.2.4-1 below. Additions
and/or alterations to this list are likely, due to the unique nature of each specific project. The
level of detail contained in the cost reports may vary depending on Programmatic management
requirements and cost analysis effort.

Section Revision:
May 2, 2016

4.2.4-2

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.4 Cost Estimating and Analysis for Operations Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Figure 4.2.4-1

Operations WBS and Budget Sample Format (For Each Physical Site Location or Decentralized
Overall Research Activity)

1.0 Project Director, Management, and Administration Office
1.1 Director’s Office
1.2 Project Management Office
1.3 Site Office
1.4 Education and Public Outreach
1.5 Safety and Environmental Assurance
1.6 Administrative Services
2.0 Science Operations
2.1 Principal Investigator(s)
2.2 Scientists
2.3 Postdoctoral Scholars
2.4 Students
3.0 Significant/Important Infrastructure Modernization, Overhaul, Upgrade, Replacement,
Expansion
3.1 Equipment
3.2 Facilities/Infrastructure
3.3 Computer Systems, Instrumentation
4.0 Facility and Equipment Operations, Maintenance, Engineering, and Support Services
4.1 Operations
4.1.1 Scheduling
4.1.2 Operating
4.1.3 Testing
4.2 Maintenance
4.2.1 Corrective Maintenance
4.2.2 Preventive Maintenance
4.3 Utilities
4.3.1 Energy (e.g., electricity, natural gas, central heating, central cooling)
4.3.2 Information Technology, Communications
4.3.3 Security
4.3.4 Water
4.4 Other/General Support Services
5.0 Contingency (If Requested)

Section Revision:
May 2, 2016

4.2.4-3

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.4 Cost Estimating and Analysis for Operations Awards
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

4.2.4.4 Operations Awards Proposals Supplementary Guidance
This section discusses additional detailed information typically needed by NSF to determine if
Recipient proposals for operations awards are allowable, allocable, reasonable, and realistic.
This information is intended to supplement the standard GAO best practices and grant
guidance from the GPG in the PAPPG. The guidance should improve project execution, clarify
NSF expectations, assist Recipients, facilitate NSF review, and prevent recurrent issues with
fewer iterative resubmissions. Recipients should note any departures from this guidance and
explain their rationale. For existing awards, the Recipient should consult with the PO.


It is the Recipient’s responsibility to manage and maintain the NSF-funded facilities,
equipment, and instrumentation used in the conduct research. However, NSF rarely
maintains ownership to major research equipment and facilities it funds. This
stewardship responsibility is necessary to protect the U.S. Government’s and the
public’s investment in these unique research facilities. In accordance with federal
guidance for property that NSF owns, leases, or otherwise manages [41 CFR 102-84
“Annual Real Property Inventory”, GSA Guidance for Real Property Inventory Reporting],
Recipients should annually provide a brief discussion, cost estimate, and actual
expenditures at a high level for the following:
o Recurring routine maintenance and repair.
o Significant infrastructure changes, including modernization, overhaul, upgrade,
replacement, and/or expansion for science facilities, equipment, utilities, and/or
instrumentation.
o Utilities (including facility operation and purchase of energy)
o General support services (such as grounds and waste management).



For property that NSF does not own, lease, or otherwise manage, Recipients should
consider federal guidance when developing their operational cost proposals.



When power costs are significant and volatile, a strategy for dealing with price
fluctuation should be developed as part of the operations plan. Other examples of items
that may require separate consideration are expendables – such as cryogens, gases and
spare parts – and ancillary equipment such as refrigerators and IT equipment.



Separate funding sources and revenue streams (e.g., visitor center fees) should be
clearly delineated.



Education and Public Outreach opportunities and associated costs should be explicitly
identified and explained.



Multiyear budgets should take inflation into account, using factors discussed in Section
4.2.2.6 above. However, when NSF budgets are flat, NSF may not be able to afford
inflationary increases in operations funding for facilities, and reductions in staff and/or
operations may be required.

Contingency, if requested, must be in compliance with Section 4.2.6 of this manual.
Section Revision:
May 2, 2016

4.2.4-4

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

4.2.5 Budget Contingency Planning for the Construction Stage
4.2.5.1 NSF Policy Positions
1. “Management reserve” is not allowable in the risk-adjusted Total Project Cost (TPC)
estimate; only “contingency.”
2. Directorates shall be responsible for the first 10% of cost overruns which exceed the
Board approved TPC.
3. At the Preliminary Design Review (PDR), projects shall have a prioritized de-scoping plan
that equates to at least 10% of the performance baseline.
4. In support of NSF’s “No Cost Overrun” policy, projects shall use a confidence level for
contingency estimates between 70 and 90 percent (under a probabilistic approach)
based on the particulars of the project and the inherent ability to de-scope.
5. NSF will hold budget contingency through project completion, in an amount up to 100%
of the total NSF-approved contingency budget, until it can be justified for obligation.
4.2.5.2 Introduction
NSF’s “No Cost Overrun” policy was originally codified in the Fiscal Year (FY) 2009 budget
request to Congress which reads:
“NSF is implementing a ‘no cost overrun’ policy, which will require that the cost estimate
developed at the Preliminary Design Stage have adequate contingency to cover all foreseeable
risks, and that any cost increases not covered by contingency be accommodated by reductions in
scope. NSF senior management is developing procedures to assure that the cost tracking and
management processes are robust and that the project management oversight has sufficient
authority to meet this objective. As project estimates for the current slate of projects are
revised, NSF will identify potential mechanisms for offsetting any cost increases in accordance
with this policy.”
The policy has been continually reinforced in subsequent budget requests to Congress and
although the wording has changed slightly, the intent has remained the same.
“Contingency” is a critical component of the comprehensive planning and execution of the
construction of large research facilities. This document describes the policies and procedures
concerning the planning, use, and oversight of budget contingency in the construction of
facilities fully funded by NSF and to the NSF-funded component of the scope when NSF partners
with other entities. It also describes the NSF’s process for assessing the sufficiency of
contingency, evaluating the effectiveness of management plans used for administration of
contingency, and NSF’s oversight role in the use of contingency funds.
This document applies only to award instruments (assistance awards or contracts) between NSF
and academic institutions or non-profit organizations. For assistance awards (CAs or grants)
with academic institutions and non-profit organizations, contingency is held by the Recipient in
Section Revision:
May 2, 2016

4.2.5-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

accordance with the Uniform Guidance (§ 200.433). Contract regulation governs the planning,
use and oversight of contingency for contracts with commercial organizations. Regardless of
where contingency is held, the requirement for a well substantiated risk assessment and
contingency estimate, as well as a robust oversight and administration is essential. Estimating
contingency and managing risk is an integral part of the project planning and execution process.
NSF positions on contingency, management reserve and de-scoping must be considered by the
Program and the Recipient as part of that process. Although strategies for other types of
contingency are mentioned here, this document is only intended to address management of
the budget contingency.
The definition of contingency varies widely among project management practitioners and
federal agencies. For NSF,1 budget contingency covers the “known unknowns” and is used to
mitigate identified cost or schedule risks as described in the Project Execution Plan2 (PEP). The
estimated risk-adjusted TPC, which is the sum of the performance baseline and the budget
contingency, is developed in accordance with the GAO Cost Estimating and Assessment Guide,3
as explained elsewhere in this manual. OMB’s cost principles in the Uniform Guidance address
budget contingency, and define it as:
… that part of a budget estimate of future costs (typically of large construction
projects, IT systems, or other items as approved by the Federal awarding agency)
which is associated with possible events or conditions arising from causes the
precise outcome of which is indeterminable at the time of estimate, and that
experience shows will likely result, in aggregate, in additional costs for the
approved activity or project. Amounts for major project scope changes,
unforeseen risks, or extraordinary events may not be included.
In contrast, “Management Reserve”4 is often used by industry and other organizations to cover
the unforeseen risks, or the “unknown unknowns.” However, NSF has no mechanism for
holding management reserve. As a result, the Directorate is responsible for the first 10% of
costs which exceed the approved TPC. To mitigate this risk, the project’s prioritized and timephased de-scoping plan must equal at least 10% of the performance baseline when established
1

NSF terminology aligns with that of AACE International, the Association for the Advancement of Cost Engineering, and of the
Project Management Institute’s Project Management Body of Knowledge (PMBOK Guide). See Section 5.2.3 for NSF definitions
of contingency and management reserve.
2

See Section 3.4 for details regarding the PEP. Note that the PMBOK guide refers to “Project Management Plan” rather than
PEP, but the NSF definition of PEP is equivalent.
3

Note that the NSF definitions and treatment of contingency and management reserves differ from those used in the GAO Cost
Estimating and Assessment Guide.
4

The GAO Cost Estimating and Assessment Guide (GAO-09-3SP, March 2009 uses the term “management reserve” for funds
held for mitigation of “known unknowns” whereas NSF uses the term “contingency.” For GAO, management reserves are
included in the budget baseline and are managed at the contractor level. The value of the contract includes these known
unknowns in the budget base, and the contractor decides how much money to set aside.

Section Revision:
May 2, 2016

4.2.5-2

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

at PDR. The ability to de-scope varies widely by project and the impacts on the eventual
scientific capabilities of the facility will also vary. The scope contingency plan should be well
considered and strive to minimize negative impacts. The Directorate may also choose to cover
the cost overrun from programmatic funding (and increase the TPC) in lieu of de-scoping if it
deems the science-support capabilities of the facility would be too severely impacted.1 See
Section 2.4.1 of this manual for required approvals.
The PEP describes a construction project’s scope, budget, schedule, and identified risks. It also
articulates the project’s plans for accomplishing the intended scope while satisfying the
constraints of budget and schedule, and managing those risks. An essential component of the
PEP is the Risk Management Plan (RMP), which describes the project’s procedures for risk
identification, analysis, monitoring, and handling (including de-scoping if required) so that the
project has a high likelihood of being accomplished within the total available budget. Budget
contingency is only one tool used to control project risk. The RMP will also include methods and
tools to manage scope contingency, schedule contingency, and provide robust risk handling and
monitoring processes. Refer to Section 5.2, Risk Management Guidelines, for additional
information.
The development of budget contingency entails estimating the future potential impacts of
identified possible adverse events to the project (i.e. risks) if those events are ultimately
realized. In accordance with the Uniform Guidance, NSF requires the use of widely accepted risk
management practices (including parametric and probabilistic methods depending on project
maturity) to estimate a range or distribution of contingency. An appropriate value is then
selected from that range that will enable the project to successfully complete the required
scope within the TPC that is sent forward for National Science Board (NSB) for approval. In
support of NSF’s “No Cost Overrun” policy, confidence levels must be in the 70-90%2 range
when the project baseline is set following PDR depending on the nature of the project;
including the ability to de-scope. This applies even for higher risk projects. The resulting TPC
estimate, including estimated contingency required, will ultimately factor into NSF’s decision on
whether or not to proceed with the project. This policy position is in no way intended to
discourage the construction of cutting-edge, high risk facilities needed to advance scientific
understanding. It is intended to give a high degree of confidence that the project will come in
on budget and clearly articulate the level of risk involved so that sound decisions can be
made. Following construction start, if subsequent analysis shows that confidence is declining
and the TPC will be exceeded, NSF requires that a reduction in scope be considered as the initial
strategy to bring the costs back in line with the budget.
1

Directorates are able to do this as a result of NSF’s “transfer authority” which is dependent on continued inclusion in the
appropriation act. The language may require that congressional appropriation committees be notified in advance of any
reprogramming. Directorates should consult with the Budget Office during the decision-making process.
2

GAO Cost Estimating and Assessment Guide (GAO-09-3SP, March 2009, pg. 158) states that the use of confidence levels of 70
to 80 percent is now common practice, particularly with projects having higher design complexity and technology uncertainty
as with NSF-funded facilities.

Section Revision:
May 2, 2016

4.2.5-3

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

Since development of contingency is statistically-based, there is a chance that not every risk
will be realized at its maximum impact. Therefore, even when properly managed, it is
possible that contingency dollars will remain at the end of the project. Once project objectives
are met and the project completed, any residual funds must be de-obligated and returned to
NSF at which time NSF will request possible re-allocation of those dollars to other agency
priorities. Awarded contingency shall be held by the Recipient until project completion, but no
later. Budget, Finance and Award Management (BFA), the Large Facilities Office (LFO), and the
Program Office will conduct a project close-out with the Recipient in accordance with NSF
policy and as described in Section 2.4.3 of this manual.
Major strategies used by NSF to ensure accountability in the management of contingency
budgets include:


Contingency budgets are developed in accordance with widely accepted standards for
risk assessment and planning. Contingency budget, scope, and schedule are similarly
derived from probabilistic, bottom-up assessments of the entire project scope.



Contingency budgets are evaluated for reasonableness by NSF through use of expert
review panels convened by the Program that examine the BOE and methodology, and
compare the cumulative contingency reserve amounts with historical experience on
similar projects. This happens at each phase of the project (Conceptual, Preliminary, and
Final Design) at increasing levels of refinement. Other divisions within NSF, and
potentially contracted experts, will also evaluate the contingency estimate as part of the
total project cost assessment as it moves through these phases.



The overall status of remaining contingency, future liens on contingency, and all
allocations and returns of contingency funds (as risks are realized or retired) are
reported on a periodic basis as specified in the award instrument. This is part of the
standard project reporting and requires archiving in the permanent electronic record
used by NSF (FastLane/e-Jacket).



Management and use of contingency is documented separately through the
configuration and change control process and must reference the associated Work
Breakdown Structure (WBS) elements and/or the previously identified Risk. The Earned
Value Management (EVM) framework for financial status reporting will eventually
reflect movement of contingency into the baseline budget (increase or decrease in
Budget at Completion; BAC). Although traceable as allocations or returns to the
contingency budget, contingency dollars become part of the baseline and are no longer
separately identifiable as contingency once incorporated.



All project expenditures must be used only for scope as defined by the elements of the
NSF approved performance baseline, and all are subject to financial audit.



Management of contingency is described in the Configuration and/or Contingency
Management Plan as part of the PEP. In this plan, thresholds are established (based on

Section Revision:
May 2, 2016

4.2.5-4

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

the nature of the project) on who has the authority to approve the use of contingency.
These thresholds are also documented in the award instrument. Below the thresholds,
the Recipient has authority to manage and allocate contingency budget to specific inscope elements of the project WBS following the Configuration Change Control Process.
Above these thresholds, approvals from NSF are required, with the level of approval
corresponding to the magnitude of the proposed change.


Financial controls prevent the cumulative Recipient cash draws from exceeding the
obligated spending authority in NSF’s financial system.

4.2.5.3 Contingency Planning and Assessment during Conceptual Design
A budget estimate, like the measurement of a physical quantity, has a value and an uncertainty
dependent on where the project is in the design process. The uncertainty in the budget
estimate is a consequence of identification of foreseen project risks and other “known
unknowns” that are under the control of the project; including scope that is not cost effective
to define in detail during preconstruction planning or the earlier phases of design. The ability to
estimate these risks and uncertainties naturally changes over time as the design is refined and
the understanding of the project matures. Recipients are required to develop methods for
qualitative and quantitative assessment of these risks, and to develop an optimized risk
handling strategy that evolves with the project.1 Regardless of the phase, the BOE for
contingency development must be sound and well documented, but remain appropriate for
that phase.
For the Conceptual Design Phase, both the performance baseline estimate and the uncertainty
of that estimate should be based on expert judgment and parametric models developed by the
project planners based on scaling and extrapolating historical data from projects with similar
characteristics. When NSF conducts the Conceptual Design Review (CDR), it expects that
Recipient will have developed a risk-based, budget contingency estimate at a similarly refined
level of detail; one that is based on estimates for major elements or functional components of
the proposed facility. NSF will conduct the CDR using a panel of experts able to apply prior
experience to assess the reasonableness of the budget and contingency estimates. The budget
contingency estimate will be evaluated by NSF as part of its first internal cost analysis for the
project based on the CDR deliverables. This initial cost analysis will help inform the cost book
and other deliverables developed during the Preliminary Design Phase.

1 See Section 5.2, Risk Management Guidelines, for more information about formulating and implementing Risk Management
Planning, and standard references on project management, such as the PMBOK Guide, for a detailed explanation of the
individual steps in Risk Management Planning: risk identifications, qualitative and quantitative risk analysis, risk handling, and risk
monitoring.

Section Revision:
May 2, 2016

4.2.5-5

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

4.2.5.4 Contingency Planning and Assessment during Preliminary Design
During the Preliminary Design Phase, NSF requires Recipients to develop budget estimates and
associated risk estimates that are “bottom up” assessments1 that consider every element of the
entire project, using as inputs the definitions of the lowest appropriate WBS elements. For each
lowest level element, the project should estimate its expected cost, excluding unusual risks or
occurrences that are outside the control of the project (unknown unknowns normally covered
by de-scoping). The project should also separately estimate, at the appropriate WBS element
for the risk described, the technical, cost and schedule risks or uncertainties using a widely
accepted method that is employed by all estimators. NSF expects to see the project utilize a
probabilistic method of calculating a range of risk exposures appropriate to the project area in
question and the maturity of the risk assessment. Expert judgment should always be applied to
both the inputs (BOE) and outputs of this process, to the reasonableness of potential cost and
schedule impacts, and to the applicability of the process to specific areas of the project. In
some circumstances, such as where specialized knowledge of a particular technical area or
market condition exists, it can be appropriate to override the outputs based on expert
intervention. Supporting documentation should clearly articulate which risks elements were
considered and how they were modified when making any adjustments to the model outputs.
It is not always realistic or even feasible to mitigate all anticipated risks. It is extremely unlikely
that typical projects will encounter all of the risks and the full extent of possible consequences
that have been identified. The contingency estimate should be appropriate to manage only the
ensemble risk, which is much more likely to occur than the sum of the individual risks. This
approach produces a more likely estimate for the TPC compared to an approach where Cost
Account Managers increase individual WBS elements to cover risk. Use of rigorous probabilistic
cost estimating methods that estimate confidence levels for the TPC (such as Monte Carlo
methods based on probability distributions for risk) are preferred and NSF highly encourages
application of these methods where practical. As a result of these estimating activities, the
project should develop the contingency estimate that provides a high degree of confidence that
the project can be completed within budget per NSF’s “No Cost Overrun” policy.
Budget, scope, and schedule risk are usually correlated to some extent. A change in scope, for
instance, may mean more costs and additional schedule. Risk analysis and budget and schedule
contingency estimation methods must consider the degree of correlation in estimating an
appropriate level of budget contingency.

1 See Section 5.2, Risk Management Guidelines, for more information about formulating and implementing Risk Management
Planning, and standard references on project management, such as the PMBOK Guide, for a detailed explanation of the
individual steps in Risk Management Planning: risk identifications, qualitative and quantitative risk analysis, risk handling, and risk
monitoring.

Section Revision:
May 2, 2016

4.2.5-6

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

Budget contingency is developed based on risk assessment of individual WBS elements, but
once defined; it loses its identification with any specific cost element and is fungible throughout
the project to manage the overall project risk. Until then, contingency is held separately from
the project baseline budget estimate1 that is used for Earned Value Management reporting, but
is included in the Total Project Cost, regardless of the award instrument.
NSF requires the PEP to contain a performance baseline that defines the project’s intended
scope, budget, schedule, risk, and management plans. The PEP will include provision of
schedule and scope contingency2 for use by the Project Manager, developed according to the
following additional considerations:
Schedule contingency: The construction schedule should be developed in the same manner as
the budget contingency estimate, following the WBS structure at the
lowest available level of detail. The project should make a technical
estimate for each task’s duration and its dependence on other tasks.
Scope contingency:

NSF requires projects to assess possible use of scope contingency and
develop a plan to make effective use of scope contingency, if necessary,
during construction. This provides the project with an additional tool to
manage the overall project given the lack of Management Reserve
within NSF.

NSF requires, at Preliminary Design Review (PDR), that the contingency budget, schedule, and
scope are the outcome of detailed planning by the project for how best to handle the various
risks that have been identified. Some risks are most effectively handled proactively by investing
in additional developmental and design activities or resources intended to prevent the risk from
occurring.
At the PDR, NSF requires a funding profile by fiscal year that includes the commitment and
obligation of funds, plus anticipated contingency needs. The profile should be a consequence of
the project’s formulation of a resource-loaded schedule for EVM reporting. Since PDR sets the
project baseline budget and informs the budget request to Congress, this allows NSF to
determine the year-by-year construction funding profile. The annual Congressional
appropriation must be sufficient to accomplish the work proposed and provide the financial
resources needed to manage the risk activities foreseen during that period.
The budget contingency estimate will be further evaluated by NSF as part of its second internal
cost analysis for the project based on the PDR deliverables. This second cost analysis will give

1

That is, contingency is not included within the Budget at Completion (BAC). TPC = BAC + contingency.

2

See Section 5.2.3 for definitions.

Section Revision:
May 2, 2016

4.2.5-7

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

assurance on the TPC brought forward to the NSB as well as help inform the cost book and
other deliverables developed during the Final Design Phase in preparation for award.
4.2.5.5 Development of the Contingency Use Process
NSF examines the RMP at PDR to ensure that the PEP describes a formal process for Change
Control1 that includes the allocation of contingency within the project during construction. NSF
approval of the RMP, including the change control process, must be documented and
maintained in the agency’s permanent record. Under the RMP, the Project Manager (or other
designated individual) should have budget authority to transfer to or from2 the contingency
category to specific WBS elements, via a process that follows the project’s Configuration
Change Control Plan. A typical change control process, for example, may involve written
application to the Project Manager by the affected Cost Account Manager(s) and formal review
and recommendation by a Change Control Board (CCB) consisting of all other system leads. The
Project Manager must have the authority to then grant the requested funds, reject the request,
or request a change in schedule, technical scope or other corrective action. All CCB change
requests are to be logged, documented, and archived by the project, with the logs and
documentation provided on a periodic, pre-determined basis to NSF for review. The defined
CCB process must include a provision for seeking prior written approval from NSF (Program
Officer or higher depending on the magnitude) for all actions that exceed the thresholds
specified in the award instrument or NSF policy.
The CCB change request document, whether forwarded to NSF for approval or not, must have
the minimum content requirements necessary to comply with relevant cost principles as well as
to maintain an audit trail. See SAMPLE CHANGE CONTROL REQUEST FORM at the end of this
section. This process must be examined by NSF for compliance before approval of the Change
Control Plan. CCB documentation shall specify all control accounts that budget is being
allocated to or recovered from, and tie to budgets itemized by cost element (i.e., labor,
materials, supplies, etc.). Contingency allocations must be supported by analysis demonstrating
that the proposed amounts to be allocated are considered reasonable and allowable.
Allocations from contingency and returns to it may have the effect of changing the baseline
budget. Therefore, it is essential that historical information be logged and maintained in a
manner that allows NSF to systematically track the evolution of the baseline from its initial
release through all subsequent changes. In other words, baseline budgets must be traceable
through historical records to the initial baseline release.

1

Section 2.4, Construction Stage, contains additional information about NSF expectations for conducting change control.

2

Some realized costs will be lower than initial estimates. Once a work package is complete, any savings should be removed
from association with specific WBS elements and added to the contingency pool available to the Project Manager.

Section Revision:
May 2, 2016

4.2.5-8

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

4.2.5.6 Contingency Planning and Assessment during Final Design
NSF requires the project to refine its cost estimates following PDR, adding additional definition
and improved confidence with the tasks associated with accomplishing the project deliverables.
At the Final Design Review (FDR) the budget estimate should be substantially based on
externally obtained cost estimates (vendor quotes, bids, historical data, etc.). This added
definition is expected to result in an increase in the project’s estimated Budget at Completion
(BAC) and a reduction in its budget contingency, while TPC remains constant. Also as part of the
FDR, NSF assesses the methodology employed by the project to further refine its cost and
contingency estimates including schedule and scope adjustments. All of this information would
then factor in to the total project cost assessment being refined and evaluated by other
divisions within NSF to make the initial construction award.
4.2.5.7 Contingency Use and NSF Oversight during Construction
NSF will negotiate the award instrument with the Recipient to fund project construction
activities (Construction Stage). This instrument will specify the contingency amounts and
include thresholds above which prior written NSF approval is required before the Project
Manager may allocate contingency (as described in the approved Change Control Process in the
PEP) to, or from, specific WBS elements.1 The thresholds will vary depending upon the
particulars of each project. Working with the Recipient, NSF will employ the following criteria
when establishing the threshold or thresholds. These considerations shall be documented in
the award file as well as the PEP and the IMP.






Award and Sub-Award amounts – A larger award amount may warrant establishment
of higher thresholds to lower administrative burden.
Sufficiency of project plans and designs – More detailed project plans, specifications
and designs generally lead to higher confidence and better bids which may allow the
thresholds to be higher.
Nature of identified project risks – The more risk associated with the nature, timing and
the severity of certain project work packages may increase the need for establishing a
lower threshold.
Review Recommendations – Expert panel findings and recommendations should be
considered in setting thresholds.
Recipient or Sub-recipient past performance history – Available past performance
information may help to indicate whether a Recipient’s change control process is
adequate or whether the Recipient has been successful in identifying contingencies,
e.g., use and accuracy of contingency logs, and therefore support a corresponding
appropriate threshold. Poor performance would support a lower threshold.

1

Thresholds are necessary to allow the project to respond in a timely way to small, immediate needs for use of contingency,
such as field changes during construction. This avoids potential cost escalation that could result from delay.

Section Revision:
May 2, 2016

4.2.5-9

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)





Known audit findings and their disposition – Relevant audit findings/dispositions
should be considered in establishing thresholds.
Sufficiency of Recipient administrative systems – The adequacy of compliance with
financial and administrative systems including accounting systems, historical cost data,
and financial reports may impact the thresholds.
Degree of NSF substantial involvement in the project – The complexity and risks
associated with the project may warrant more NSF involvement and hence lower
thresholds.

Once construction begins, the actual cost for some specific WBS elements may exceed the
estimated cost and the Project Manager can choose to allocate contingency in accordance with
the process defined in the PEP for Change Control. In other cases, the actual cost will be less
than the estimates, and the Project Manager may decide to transfer budgeted funds from the
affected WBS elements to contingency. In case, whether it’s a risk realized or a risk retired, the
Change Control documentation must tie this transfer back to an identified risk element in the
Risk Management Plan to be allowable.
Contingency funds are to be used only to support scope that is part of the NSF-approved
project baseline, as defined in the PEP and successive CCB actions. Depending on the
thresholds, Project Manager, CCB, NSF, and NSB approvals are required to modify the project
scope. Unexpended contingency funds may not be used to support operations or other outof-scope activities.
4.2.5.8 Reporting Requirements
Each project in construction must report monthly to NSF on the status of the project, while
projects in the Design stage are highly encouraged to submit a monthly report. At a minimum,
the monthly report will include: (1) the amount of available budget and schedule contingency,
as a total amount, and as a percentage of the estimated cost to complete (ETC) the project; (2)
updated projections for major milestones and project end dates; (3) an updated BAC and Estimate
at Completion (EAC) for each second level element; and (4) an updated change log indicating all
contingency allocations (“puts and takes”) and a “liens” list of projected amounts of possible
future calls on contingency. Projects are expected to periodically compute and update the risk
exposure, ETC, and EAC, and compare these quantities to the BAC and TPC. NSF will monitor
the financial information provided and compare the available contingency to the estimated
remaining risk exposure. NSF may request corrective action if the contingency budget appears
inadequate to manage remaining risk.
All CCB actions, irrespective of amount, or whether they increase or decrease the BAC, must be
reported directly to Program Officer at least quarterly. All CCB actions exceeding defined
thresholds for allocation of budget, schedule, or scope contingency shall be approved by NSF as
codified in the PEP and the CA. NSF-approved CCB actions must be made part of the award’s

Section Revision:
May 2, 2016

4.2.5-10

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

permanent record. For assistance awards (CAs or grants), CCB documentation is maintained in
NSF’s electronic record system (e-jacket) in accordance with the award terms and conditions.
NSF’s financial system controls prevent the cumulative Recipient cash draws from exceeding
the obligated spending authority. All funds are retained within NSF’s obligated award amount
to be drawn down by the Recipient for allowable expenses once needed. NSF conducts various
post-award monitoring activities, such as periodic external reviews (whose scope includes
financial as well as technical status), site visits, and single and program-specific audits to
monitor compliance.
4.2.5.9 Partnership Considerations
NSF may partner with other entities to plan and construct a major facility. The guidelines within
this document are applicable when NSF funds a particular scope of work within a larger overall
project. Risk assessment and contingency development processes are to be applied to those
WBS elements funded by NSF. Similarly, the Recipient managing construction must report on
the use of contingency during construction in accordance with the requirements regarding use
of contingency funds.
More complex situations may arise when NSF funds a proportion of the total project cost, or
where NSF contributes along with others to a common fund to build specific WBS elements
within the context of a larger project. Because overall project risk is reduced as more WBS
elements are aggregated into the risk analysis and managed through a centrally held
contingency fund during construction, NSF encourages the development of unified
management for project planning and execution of the entire project scope wherever practical.
However, NSF recognizes other partners may have different processes for planning, funding,
and conducting oversight, making it challenging to form a unified management structure.
Consequently, the award instrument must define the specific procedures for handling
contingency in those circumstances. Program Officers are advised to consult with the Division
of Acquisition and Cooperative Support to determine an effective approach consistent with the
principles of federal laws and regulations. The Large Facilities Office may be able to provide
models of various approaches that have been used successfully in other projects.

Section Revision:
May 2, 2016

4.2.5-11

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)
Figure 4.2.5-1

Section Revision:
May 2, 2016

Sample of a Change Control Request Form, with instructions for filling out the various sections

4.2.5-12

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

Section Revision:
May 2, 2016

4.2.5-13

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.5 Budget Contingency Planning for the Construction Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

Section Revision:
May 2, 2016

4.2.5-14

Large Facilities Manual: NSF 16-0XX (May 2016)
4.2.6 Budget Contingency Planning during the Operations Stage
Prepared by Budget, Finance, and Awards Management, Division of Acquisition and Cooperative
Support (BFA-DACS), & The Large Facilities Office (BFA-LFO)

4.2.6 Budget Contingency Planning during the Operations Stage
Any request for contingency must comply with paragraph 200.433 of the Uniform Guidance. As
a result, it is generally more appropriate for operating budgets to include only explicitly
identified allowances for repairs, maintenance and other factors such as “technology refresh”
for cyber-infrastructure or other similar up-grades. Unless a formal risk assessment is
conducted and a Risk Management Plan for operations implemented, it is recommended that
each project have in place a systematic program to identify the potential cost and operations
impacts of both recurring and non-recurring events to develop these allowances and include
this information as part of the operating plan.
A Program Officer (PO) may request a periodic formal Condition Assessment report (an
evaluation of capital assets requiring significant expenditures for periodic replacement or
refurbishment and having a lifetime longer than the usual five-year award cycle), accompanied
by an Asset Management Plan (a strategic plan for dealing with these issues), to inform NSF and
the facility management of anticipated major and infrequent maintenance expenses that cause
a significant departure from the routine funding profile. This allows NSF, as part of its budget
allocation process, to proactively address these issues before they become immediate needs.
Operating budgets should include, when appropriate, resources to provide a continuing
program of advanced research and development (R&D) that will enable a facility to evolve its
scientific program and best meet the needs of the research community. Funding for these kinds
of up-grades may also come from separate equipment and/or instrumentation programs within
the Directorate or Division. The PO should be closely involved in monitoring and assessing the
facility’s evolution and in supporting advanced R&D planning and budgeting. Evaluation of each
large NSF facility, as part of its yearly operations review, should include a section on the plans
for advanced R&D and should relate these plans to the anticipated evolving mission of the
facility. This evaluation helps guide the PO in formulating a budget strategy for funding
advanced R&D efforts.
It is important that NSF identify and devise plans to address the specific issues that arise as part
of the divestment and closeout of a facility at the end of its scientifically competitive life. It is
recommended that the PO develop a process for projecting the anticipated divestment of the
facility along with the costs and legal requirements of this action. For example, annual review of
an evolving plan for the decommissioning and disposal of the facility assets and environmental
obligations needs to be systematically considered as part of the facilities operations mission.
This process should create and keep current a plan for the facility’s divestment and closeout,
along with its associated budget liability. While not part of the annual budgeting process, this
information informs the longer-term strategic planning at the NSF Division and Directorate
levels.

Section Revision:
May 2, 2016

4.2.6-15

Large Facilities Manual: NSF 16-0XX (May 2016)
4.3 System Integration, Commissioning, Testing and Acceptance
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

4.3

SYSTEM INTEGRATION, COMMISSIONING, TESTING AND ACCEPTANCE

System integration, commissioning, testing and acceptance are recipient functions, and are an
essential part of complex construction/acquisition projects. Failure to perform them, or to
adequately plan for them, can lead to serious cost and schedule overruns. The recipient is
required to describe its plans for system integration, commissioning, testing and acceptance in
the PEP. The Program Officer (PO) approves these plans, but is also required to include periodic
review of progress in these areas:






System Integration – combining and coordinating the many physical and performance
interfaces in a project;
Commissioning – substantiating the capability of the facility to function as designed by
bringing various system components on line first sequentially and then in simultaneous
operations to study and affirm the interaction among subsystems;
Testing – assessing the operation of the facility by applying the criteria established in
the PEP to measure acceptable performance; and
Conditions for Acceptance – specifying the expected condition of the facility, its
performance attributes, the tests the recipient will perform, and the data it will consider
prior to accepting the facility or components of the facility and declaring it ready for
Operations and Maintenance. In some cases, a phased approach to acceptance will be
required. For example, for distributed-but-integrated facilities or for facilities with
complex instrumentation and equipment, the PO will want the recipient to demonstrate
performance and perform acceptance procedures for part of the system prior to
proceeding with construction and/or acquisition of other systems. The PO, in
consultation with the Integrated Project Team (IPT), will determine whether the
recipient will conduct the tests and accept the facility or whether the PO will participate
in the testing and accept the facility on behalf of the government.

Frequently, some aspects of construction and/or acquisition overlap with initial operation. A
detailed Transition Plan should be developed by the Recipient and incorporated into the PEP at
least one year prior to the anticipated commencement of commissioning activities. Elements of
the Transition Plan are first addressed during Conceptual Design, and become progressively
more detailed as planning evolves. During construction, the PO reviews the plan, utilizing
internal staff, external experts, consultants, external review panels and the resources of the
Large Facilities Office.1 The review of the Transition Plan considers the following questions:


1

Will the project have parallel periods of construction/acquisition and operations, with
some components coming on line earlier than others?

Optional for projects not constructed with MREFC funds.

Section Revision:
June 17, 2015

4.3-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.3 System Integration, Commissioning, Testing and Acceptance
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)









What is the project’s strategy for facility acceptance, operational readiness review, site
safety and security, and training of operational staff and members of the research
community utilizing the facility?
What are the project plans for transitioning staff from construction to operational
support activities? Is there a plan to bring in personnel with the requisite technical skills
to operate and support the facility at appropriate times? Have training needs been
addressed?
What risks to the project might result from contractor interference during periods of
beneficial use or occupancy as construction activities conclude?
What contracting strategies are employed to ensure that priority tasks are completed in
a timely way and do not delay operational readiness?
What are project plans for obtaining use and occupancy permits, or satisfying other local
regulatory criteria?
Do the budgets reflect a proper allocation between construction/acquisition and
operations?

For projects funded through the MREFC account, even if limited operations are undertaken, the
changeover from MREFC funding to Research and Related Activities (R&RA) and/or EHR funding
does not have to occur until the facility has been accepted and the PO ensures that the budget
is estimated accordingly. Where R&RA and/or EHR funding will be used prior to acceptance, the
PO will ensure that the budget justification clearly describes the changeover and that the
earlier changeover is estimated and budgeted accordingly.

Section Revision:
June 17, 2015

4.3-2

Large Facilities Manual: NSF 16-0XX (May 2016)
4.4 Documentation Requirements
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

4.4

DOCUMENTATION REQUIREMENTS

The Recipient is responsible for ensuring that a document management system is in place that
provides for retention and retrieval of essential and significant documentation related to the
project. Recipient documentation may take many forms, from informal e-mail communications
to formal letters, bids and contracts. NSF strongly prefers that this system be electronically
accessible via Internet, rather than paper-based, but recognizes that some paper records are
necessary. The documentation system should not only aid in identifying the types of documents
to retain, but should also contain appropriate controls over official documents such as drawings
to ensure that only the most recent drawings are being used and that only authorized
personnel are able to access and modify them. A sound document management system will
help prevent miscommunications and misunderstandings and will ensure that the facility
operators have the information required to maintain the facility.
Recipients should retain financial records, supporting documents, statistical records and other
records pertinent to the award instrument (CAs, grants, subawards, and subcontracts) for a
period of three years after submission of the Final Project Report. In addition, access to any
pertinent books, documents, papers and records should be made available to the NSF Director
and the Comptroller General of the United States or any of their duly authorized
representatives to make audits, examinations, excerpts and transcripts in accordance with
either the Uniform Guidance or FAR requirements.
The documentation required, and the responsibility for producing and maintaining it, varies
within the facility life cycle. During the Design and Development Stage, the Program Officer
(PO) is responsible for producing and maintaining documentation related to review and
approval of awards. Managing the documentation pertaining to the review and processing of
proposals and awards is the PO’s responsibility throughout the life of the project. Chapter VI of
the Proposal and Award Manual (PAM) requires that proposal decisions be clearly documented.
Chapter XII of the PAM requires that NSF award records be retained and either retired or
disposed of in accordance with Federal law and regulation. NSF documentation should include
all partnership and other agreements, standard eJacket submission in the NSF-required format,
the Internal Management Plan (IMP), the Baseline Project Definition (typically defined in the
PEP), the record of oversight (including all reviews and reports), and all significant project
correspondence.
During the Construction Stage, essential and significant documentation includes the record of
any decision affecting the cost, schedule or performance baseline. At a minimum, the following
forms of documentation should be retained:





Memorandum of Understanding (MOU) and any other project agreements or deals;
Architectural, engineering, shop and as-built drawings;
Correspondence identifying problems, the resolution process, and the final decision;
Contingency use log;

Section Revision:
May 2, 2016

4.4-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.4 Documentation Requirements
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)




Change requests and approvals; and
System integration, commissioning, testing and acceptance plans and results.

During the Operations & Maintenance Stage, the Recipient documents facility performance in
terms of:



The facility itself – e. g., historical record of all costs related to maintenance (preventive,
deferred, repairs and/or emergency), operating time, and scheduled as well as
unscheduled downtime, and
Use of the facility for research and education (including a record of users that includes
the name, affiliation, funding agency, award number and annual award amount for each
user).

Section Revision:
May 2, 2016

4.4-2

Large Facilities Manual: NSF 16-0XX (May 2016)
4.5.1 Introduction to Oversight, Reviews, Audits and Reporting
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

4.5

REQUIREMENTS FOR NSF PERFORMANCE OVERSIGHT, REVIEWS, AUDITS, AND
REPORTING

4.5.1 Introduction to Oversight, Reviews, Audits and Reporting
Oversight, reviews, audits, and reporting requirements change as a facility moves through its
life cycle and differ substantially between the Design, Construction, and Operations Stages. The
Recipient is responsible for complying with the reporting requirements contained in the award
instrument (e.g., technical and financial reporting), this manual, and in the Proposal and Award
Policy and Procedures Guide (PAPPG) – particularly with respect to property management and
final reporting and closeout requirements for termination of the award. The Recipient is also
responsible for providing internal oversight of its own activities. This may require internal
reporting and reviews by committees established by the Recipient institution for the purpose of
oversight. NSF is responsible for reporting its performance goals for construction projects based
on Recipient EVM reports, per the Government Performance and Results Act (GPRA) of 2010.
Reviews and reporting are an important part of the oversight process that allows the PO to
monitor performance and compliance with project goals. Due to the complex nature of
facilities, the level of oversight will be considerably greater than for a typical NSF research
grant. The Program Officer (PO) has continuous responsibility for oversight of the facility in
accordance with the Internal Management Plan (IMP) and through various reviews and reports,
such as consultation and coordination with the Large Facilities Office, coordination of assurance
through the NSF Integrated Project Team (IPT), and periodic updates to the MREFC Panel (if
applicable) and the NSB.
Reviews and reporting incur certain costs. Depending on the size of the project and the
distribution of the information, these costs may be significant enough to warrant explicit
inclusion in the project budget. Review and reporting plans and costs should be identified in the
PO’s IMP and in the Recipient’s PEP so that they can be adequately considered in the project
budget and schedule. The PO should clearly define the reporting requirements that are the
responsibility of the Recipient in the award instrument and these requirements should be noted
as milestones on the project master schedule for construction. The Recipient’s Project Director
adheres to their internal practices regarding financial and business operations controls,1 and
internal reporting (e.g., to the Principal Investigator, Dean, etc., as applicable and required).
It is important that consideration be given to Conflict of Interest rules and Privacy Act
restrictions when distributing and sharing reports containing proprietary or confidential
information.

1

See NSF "Business Systems Review (BSR) Guide.”

Section Revision:
May 2, 2016

4.5.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.5.2 Recipient Performance Reports
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

4.5.2 Recipient Performance Reports
Reporting requirements vary by project life-cycle stage (Design, Construction, Operation, and
Divestment) and are specified, either explicitly or by reference, in the Terms and Conditions
associated with the grant or award instrument between NSF and the recipient. Performance
reports are generally provided on a monthly and/or quarterly basis, with a comprehensive
annual report provided by a predetermined date. Separate reporting activities are required for
MREFC funds used for construction activity if the facility also receives Research and Related
Activities (R&RA) funds. These are specified in the Terms and Conditions of the separate
cooperative agreements for each type of funding.
NSF may occasionally request that Awardees provide additional information on a specific
activity, often as a result of a unique request from within the government. Such requests may
not be specifically included in the cooperative agreements due to their ad hoc and individual
nature, nevertheless an awardee must be prepared to respond. Examples include responses to
audit requests from the Office of Inspector General and queries from the US Congress and
Executive Branch agencies. Some projects, particularly those with construction activities or
frequent changes in design, will need more frequent reporting intervals. For example, providing
the written minutes from a weekly construction meeting is common practice.
During the Construction Stage, the Project Director, who is responsible for executing and
controlling the project in accordance with the PEP and the award instrument, reports to the
Program Officer (PO) on a periodic basis (monthly for MREFC-funded projects and no less than
quarterly in other cases). Those reports should include the following:








Summary of financial and technical status – work accomplished during the reporting
period, including major scientific and/or technical accomplishments and milestones
achieved;
Comparison of actual cost and schedule to planned cost and schedule, at least at the
second level of detail in the WBS, using Earned Value Management System (EVMS)
methodology;
Variance reports at an appropriate cost account or work package level for all cost and
schedule variances > ±10%, including explanation of causes and remediation/recovery
plans1;
Status of cost and schedule contingency and risk exposure, as listed in Section 4.2.5.8
Reporting Requirements;
Review of current or anticipated problem areas and corrective actions;
Management information such as changes in key personnel, sub-contracts/awards and
sub-contractor/recipient performance, as well as any other information about which the
PO needs to be aware; and

1

Variance reports provided by recipients are used by NSF in its metrics for construction project performance goals, In
accordance with the GPRA Modernization Act of 2010. See Section 4.5.4, NSF Performance metric for Construction.

Section Revision:
May 2, 2016

4.5.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.5.2 Recipient Performance Reports
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)



Concerns, upcoming milestones, or project deliverables.

For MREFC projects in the Construction Stage, the PO is responsible for providing the LFO a
written monthly summary of this information in a standard format provided by the Head, Large
Facility Office (HLFO). Smaller-scale projects that are not funded through the MREFC account
will provide status reports to the PO with a frequency and level of detail defined in their
respective Internal Management Plans (IMPs). In every case, the PO is responsible for keeping
the appropriate NSF staff (Grants and Agreements or Contracting Officer, Division Director (DD),
Assistant Directors (ADs), Integrated Project Team [IPT] members, etc.) informed of the project
status.
In executing and controlling the project, the Recipient manages the project to the Baseline
Project Definition and cost and schedule. Per Section 4.5.4 NSF’s Performance Metric for
Construction, the Recipient will notify the PO of total project cost and schedule variances
exceeding -10%, including explanations of causes, recovery plan, and timeline for correction, as
part of the routine reporting process.

Section Revision:
May 2, 2016

4.5.2-2

Large Facilities Manual: NSF 16-0XX (May 2016)
4.5.3 Recipient Performance Reviews and Audits
Prepared by the Large Facilities Office (BFA-LFO) and the Cooperative Support Branch (BFA-CSB) in
the Budget, Finance, and Award Management Office

4.5.3 Recipient Performance Reviews and Audits
4.5.3.1 Recipient Internal Reviews
The Recipient will demonstrate appropriate internal management by conducting reviews and
internal audits in conformance with generally accepted accounting and project or operations
management practices and standards. These reviews include, but are not limited to, technical
merit reviews, design reviews, procurement readiness reviews, progress and planning reviews,
risk reviews, safety reviews and walkthroughs, acceptance reviews, and self-audits. The kinds
and frequency of all reviews and audits should be addressed in the Project Execution Plan.
Although internal review team are typically populated by project staff, consideration should be
given to the inclusion of outside subject matter experts on review teams, who can provide a
valuable independent perspective.

4.5.3.2 NSF External Reviews
NSF requires periodic external reviews that provide advice on the status and anticipated future
performance of the facility activities. The frequency and content of these reviews are specified
in the terms and conditions of the award instrument. Typically, Annual Operations or Project
Reports and Final Project Reports, or activities with the equivalent purpose, are required to
evaluate progress and provide feedback. Additional ad hoc reviews may be requested by the PO
under certain circumstances, such as significant re-planning of construction projects, changes in
key personnel, and major changes in research technical design, direction, or scope. These
reviews should determine the extent to which the facility is meeting the goals of their Annual
Plan as well as the overall goals for the award, discuss any upcoming challenges and highlight
best practices that could be applied to other large NSF facilities. Whenever possible, the review
should be conducted at the facility itself by an external panel with expertise in the construction
and operations of large scientific facilities.1 The panel should produce a formal written report.
The review is organized and conducted by the PO in consultation with the LFO Liaison. Both the
review committee membership and the charge to the committee require concurrence from the
PO and LFO Liaison. Invitees to the review shall include the PO, the cognizant Business Officer,
and staff from the Large Facilities Office. Following the review, the LFO Liaison will produce an
independent assessment of the review for the HLFO. The Operations or Performance Review is
not meant to compete with other reviews listed in Section 4.5.3., such as the Business Systems
Review (BSR), which looks at business processes.
Careful consideration should be given to the selection of independent reviewers, and in all
cases the skill sets of the reviewers should be matched to the type and kind of review to be
conducted. Broad programmatic review panels charged with reviewing all aspects of a project
will generally have representation from the academic and broader national/international
1

Consult with the NSF Program Officer for f guidance and best practices with respect to planning and executing External
Reviews of NSF’s Large Facilities.

Section Revision:
May 2, 2016

4.5.3-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.5.3 Recipient Performance Reviews and Audits
Prepared by the Large Facilities Office (BFA-LFO) and the Cooperative Support Branch (BFA-CSB) in
the Budget, Finance, and Award Management Office

research community, as well as experts in administrative aspects of facilities/project
management. A review panel focusing on specific administrative or technical aspects of a
project would have a different set of skills.
The PO will typically use a standard review “template.”1 These well-defined review formats
provide a broad outline against which the project can be compared and checklists that can be
used to assess the status of the project. These reviews can be particularly helpful in the preaward phase in ensuring that the project is ready to be implemented. Exceptional
circumstances may arise that necessitate some alternate format. In this case, the PO consults
with the LFO Liaison to constitute a review charge and format tailored to meet the specific
requirements of the review.

4.5.3.3 Business Systems Reviews (BSR)
The BSR is one of NSF’s advanced monitoring activities that assist with oversight and provide
assurance of the suite of business systems that support the administrative management of a
large facility. These reviews are designed to provide reasonable assurance that the business
systems (people, processes, and technologies) of NSF Recipients are effective in meeting
administrative responsibilities as well as satisfying other federal requirements. Specifically, a
BSR verifies that administrative and financial policies and procedures are written; determines if
these policies and procedures conform to OMB requirements, NSF expectations, and other
applicable federal regulations; and if they are used to administratively manage the large facility
in each of the core functional areas. BSRs are also intended to provide an opportunity for crossfertilization of ideas through the identification of best practices, and serve to refocus recipients
on the importance of administrative quality.
The LFO has the lead role in coordinating the assessment of these systems by using desk
reviews and site visits to determine if the administrative business systems used in managing the
facility meet NSF expectations and are in compliance with federal regulations.
BSRs are generally conducted on a five-year cycle for all facilities. For new MREFC projects, the
BSR is ideally conducted prior to the construction award. The LFO uses an internal risk
assessment process to decide which facilities will have a BSR conducted in any given year. Risk
factors include:




1

When the last BSR on the facility was conducted;
New MREFC project being proposed;
Other identified risks such as audit finding, significant changes in funding levels,
management, scope, operational performance or mission of the large facility;

Please contact the cognizant NSF PO for details and a description of best practices and/or preferred templates.

Section Revision:
May 2, 2016

4.5.3-2

Large Facilities Manual: NSF 16-0XX (May 2016)
4.5.3 Recipient Performance Reviews and Audits
Prepared by the Large Facilities Office (BFA-LFO) and the Cooperative Support Branch (BFA-CSB) in
the Budget, Finance, and Award Management Office

Further information and various details of the BSR process are provided in the BSR guide, 1
which defines and establishes the procedures for the planning, execution and follow-up
activities associated with conducting BSRs. The Guide also defines the roles and responsibilities
of NSF staff assigned to BSR activities and identifies core and targeted review areas.
4.5.3.4 Cost Incurred Audits
NSF conducts a cost incurred audit for large facility awards above $100M at the end of the
award and potentially during execution of the award based on an annual large facility risk
assessment conducted by the Large Facilities Office and the Cooperative Support Branch at
NSF. Recipients should be prepared for such an audit at any time based on 2 CFR 200.205-7 of
the Uniform Administrative Requirements, Cost Principles and Audit Requirements for Federal
Awards and as stated in the terms and conditions in the Cooperative Agreement.
Recipients must maintain adequate internal controls, policies and procedures, and reliable
accounting systems. The purpose of the Cost Incurred Audit is to provide prudent oversight for
those responsible for the effectiveness, efficiency, and economy of the recipients' operations
and the use of Federal funds. The Incurred Cost Audit is performed to assure the existence of
adequate controls which will prevent or avoid waste, fraud, and abuse and inefficient practices.
In preparation for a cost incurred audit, recipients are required to submit financial expenditures
(incurred cost) data to NSF on a frequency to be determined by the Grants and Agreements
Officer and the Cooperative Agreement’s Terms and Conditions. It will be no less frequently
than annually. The Large Facilities Financial Data Collection Tool, created by NSF to assist
recipients in preparing and recording financial expenditure information for its cooperative
agreements for large facilities, is required for submission of the financial expenditures data. The
Large Facilities Financial Data Collection Tool is a macro-enabled Excel workbook that provides
recipients a single, standardized method for submitting direct and indirect expenditure data
with minimal effort and help to ensure data quality and accuracy. It will enable NSF and
independent auditors to more easily and consistently collect the financial data required to
maintain proper oversight and stewardship over funds and facilitate recipient compliance in
meeting its reporting requirements.
To complete the data collection, recipients will need to collect expenditure information for all
active Cooperative Agreements during the awards performance period. Suggested sources
from which to collect this data include the recipient’s Chart of Accounts, General Ledger,
Project Cost Ledger, NSF Award budget, and the award organization’s Negotiated Indirect Cost
Rate Agreement. To submit expenditure data, recipients can download the Large Facilities
Financial Data Collection Tool file from NSF at: http://www.nsf.gov/bfa/lfo/lfo_documents.jsp

1

See "Business Systems Review (BSR) Guide.”

Section Revision:
May 2, 2016

4.5.3-3

Large Facilities Manual: NSF 16-0XX (May 2016)
4.5.3 Recipient Performance Reviews and Audits
Prepared by the Large Facilities Office (BFA-LFO) and the Cooperative Support Branch (BFA-CSB) in
the Budget, Finance, and Award Management Office

Once downloaded, recipients should save the file and then begin to enter data following the
guidance provided in the tool. See Figure 4.5.3-1 for a flow diagram for data entry. Recipients
are instructed to submit this data collection annually to NSF as well at the end of your project
or based on the Terms and Conditions of the award. When completed, this Large Facilities
Financial Data Collection Tool will allow submission financial data electronically to the NSF
Grants and Agreements Officer or an independent auditor. Questions regarding this collection
should also be directed to the responsible NSF Grants and Agreements Officer.

Figure 4.5.3-1

Large Facility Financial Data Collection Tool Process Flow Diagram, identifying the core data
sources recipients should utilize to complete the worksheets to accurately capture the data.
Progressing through the tool's worksheets from left to right will streamline the data capture
and submission process.

4.5.3.5 Earned Value Management Validation
(This section reserved for future content)

Section Revision:
May 2, 2016

4.5.3-4

Large Facilities Manual: NSF 16-0XX (May 2016)
4.5.4 NSF Performance Metric for Construction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

4.5.4 NSF Performance Metric for Construction
In accordance with the GPRA Modernization Act of 2010 (P.L.111-352); Empowering the Nation
Through Discovery and Innovation: NSF Strategic Plan for Fiscal Years (FY) 2011-2016; and OMB
requirements, NSF developed goals to measure construction/upgrade performance based on
EVM systems used to monitor project cost and schedule. For MREFC facilities1 that are under
construction and that are more than 10 percent complete, the NSF performance metric goal is
that total project EVM cost and schedule progress variances against the performance baseline
will be kept at, or less than, negative 10 percent. (Projects that are less than 10 percent
complete are not held to this goal because EVM data is less meaningful statistically in the very
early stages of a project.)
Negative total project variances exceeding 10 percent should be reported to NSF and be
accompanied by an explanation and a proposed plan and timeline for recovery or
accommodation of the cost and schedule shortfalls (e.g., use of contingency, de-scope).2

1

This includes facilities whose construction is partially supported by funds or in-kind contributions from outside agencies. In
such cases, the variance for the total project and the variance for the MREFC-funded portion should be reported separately. For
example, if the total project variance is -7% and the MREFC portion is -12%, then the -12% would be reported, accompanied by
a recovery plan. Alternatively, if the total project variance is -15% and the MREFC variance is -11%, then both variances would
be reported with appropriate recovery plans with timelines.
2

See section 4.5.2 for requirements on variance reporting at lower WBS levels for monthly progress reports.

Section Revision:
May 2, 2016

4.5.4-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.5.5 Re-Baselining
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

4.5.5

Re-Baselining

If maintaining the original performance baseline (scope, total project cost, or end date) is no
longer possible, the Recipient will consult with the PO to determine whether re-baselining the
project is warranted. When deciding which course of action to pursue, the PO will need to
balance the effect of failing to achieve the project’s performance goals against the impact on
the research and education proposed for the completed facility.
The PO should consult with the NSF Integrated Project Team and the Directorate/Division
Leadership, prior to authorizing re-baselining a project. Variances may result from many
factors – for example, inadequate project planning or management, or factors not within the
Project Director’s (or manager’s) control. Examples of the latter include failure to identify the
complexity in particular tasks (such as integration), failure to budget for adequate labor,
materials or time versus unexpected increases in the cost of labor and/or materials,
unavailability of labor and/or materials, unusually severe weather, etc.
For construction projects, uncertainties are normally managed through re-planning1 and the
use of contingency, per Section 4.2.5. Re-baselining for construction projects occurs for
variances that result in:
1. Increases in the NSB-approved Total Project Cost (TPC),
2. A change in the approved project end date, and/or
3. Major changes in scope.
The LFO, the MREFC Panel, and the Director should be kept informed of any pending
re-baselining discussions. NSF approvals are required per Section 2.4.1 of this manual. If only
the schedule is extended without an increase in TPC, the terms and conditions of the award
instrument apply (i.e. NSF policy on No-Cost Extension for CAs). Once a re-baselined Project
Definition has been approved, the re-baselined requirements replace the Baseline Project
Definition as the standard against which progress is measured. Consequently, costs exceeding
budgeted amounts in the initial Baseline Project Definition are not referred as “overruns” once
a new project baseline has been implemented by the project management and accepted by
NSF.

1

See Section 2.4.1 for the definitions of “re-baselining” and “re-planning”.

Section Revision:
June 17, 2015

4.5.5-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.6.1 Partnerships Overview
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

4.6

PARTNERSHIPS

4.6.1 Partnerships Overview
For both MREFC and smaller projects, partnerships are an essential consideration – beginning
at project inception. Partnerships may take many forms, but typically include coordinated
funding from states or state institutions, other federal agencies,1 non-governmental entities,
and foreign funding agencies. International partnerships are generally the most complex.
Key issues in these partnerships, whether international or the result of interagency or state
collaboration, present several important challenges that the recipient and PO need to consider
carefully.
The first is “culture shock.” The science or engineering cultures in different countries will
generally exhibit great variations in procedures when it comes to funding, managing and
overseeing, constructing and operating a facility. Differences often include lack of mutual
understanding or considerably different contexts for defining the role and function of project
management. It is typically very challenging for each nation to manage its part of the project
unless there is a means for integrated management and oversight by the central Project
Manager.
The Project Director or Manager should be in place before funds are released and, to be most
effective, should be given budget authority (or authority over in-kind resources) and should not
simply act as a coordinator. In terms of oversight, reviews of project status by U.S. agencies are
not universally accepted. U.S. agencies use reviews heavily, but not all countries do. In some
countries, reviews that uncover problems may be received without a sense of urgency and may
not be acted upon quickly. U.S. partner agencies may be able to insist upon resolution of issues
when playing a majority role in funding; if not, other steps should be taken. Full project
transparency is essential to success.
A second important issue is early negotiation with international partners. There is a need to
start with a clear understanding by all partners as to how the construction project is to be
managed and the facility is to be operated. It is also important to know how agencies in
different countries view the project in terms of shared goals, the science or engineering case for
the project, and its priority. If participating partner countries all rate the priority of a project at
the highest level, then commitments carry more weight.
Funding risks associated with international partnerships should be assessed and contingency
plans developed regarding potential changes in commitment. Finally, early negotiation also

1

See “Best Practices for Federal Research and Development Facility Partnerships,” IDA Science & Technology Policy Institute,
IDA Paper P-5148 Log: H 14-000676, for guidance or models on forming interagency federal partnerships.

Section Revision:
June 17, 2015

4.6.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.6.1 Partnerships Overview
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

provides a means to establish and maintain regular agency-to-agency contacts, providing an
early understanding of funding pressures and other emerging pressures in each country.
The NSF Office of International and Integrative Activities (OIIA) should be advised of potential
international partnerships early in the process and kept apprised of significant developments.
That Office can facilitate coordination with the Department of State and the White House Office
of Science and Technology Policy on foreign policy and geopolitical issues, advise on
interactions between NSF and counterpart funding organizations in other countries, and
provide information/contacts on matters such as visa issues for project participants and cost
issues related to assessment of import duties on internationally shipped items.

Section Revision:
June 17, 2015

4.6.1-2

Large Facilities Manual: NSF 16-0XX (May 2016)
4.6.2 Partnership Funding
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

4.6.2 Partnership Funding
Funding of projects involving partnerships is obviously a central consideration. International
partner agencies need to understand the funding processes in the different countries involved.
The complexity of the NSF process can lead to misunderstandings regarding the schedule of
funding and project approvals. Because of the great variation among countries as to how labor
costs are counted, it is good practice to adopt standard costing techniques for equipment,
labor, commissioning and operations. MOUs need to be developed, detailing the foreign
contributions. In some cases, these contributions may be in cash or in-kind level of effort; but
deliverables should be clearly specified and the contributions should be valued in U.S.
equivalent terms (including all labor costs) for projects in which NSF is the lead agency. To aid
project management and eventual close-out, it should also be made clear what scope NSF and
the other partner are either paying for or contributing (by WBS element) and proper
segregation of funding rules employed as appropriate.
As with all such projects, contingency funds (or their equivalent) need to be identified by all
partners. There is great variation in practice among countries, again because labor costs may or
may not be included in contributions to the project. This can have a great impact. For example,
in a cost-overrun situation it may become expeditious to simply stretch the project out. This
may work for one country, resulting in less focus on schedule issues; but it generally does not
work for U.S. projects where “standing army” costs are directly allocated to the total cost for
construction of a facility.
In addition, when partner funding is in cash, variations in exchange rate can have a large effect
on the ability of a given country to meet its commitment on deliverables. Therefore, scope
contingencies need to be explored. When international partners do not include adequate
contingency, and the U.S. does, funding “caps” (agreed upon in advance) are an appropriate
policy. Although caps may enforce discipline, they may have other effects. For example, when
there are schedule slips and “standing army” costs rise, caps can limit the deliverables that may
be provided. Strict adherence to caps may therefore compromise the overall performance
goals.
Finally, a facility’s project management and operations plans should be well understood by all
partners. When different countries have responsibilities for separate subsystems, strong system
integration and comprehensive interface documents become very important. The changecontrol process needs to be clearly understood. Change control is made very complex because
performers in one country may be ill equipped to handle or adapt to required changes. It is also
very important to establish a sound schedule baseline and adhere to it.

Section Revision:
June 17, 2015

4.6.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
4.6.2 Partnership Funding
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

For partnerships with organizations or agencies in the United States, the following activities are
advised:





Evaluate NSF’s role (NSF’s authority and responsibility vary depending on its status as
executive agent or as a majority, equal or minority partner). Assess risks and develop a
plan to address them, e.g., implementation of controls that limit NSF’s exposure to
overruns (see Section 5.2, Risk Management Guidelines).
Ensure that all partners understand the review and approval processes of the other
partners.
Prior to entering into a partnership, develop and execute an MOU.

Section Revision:
June 17, 2015

4.6.2-2

Large Facilities Manual: NSF 16-0XX (May 2016)
4.6.3 Memorandum of Understanding (MOU)
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

4.6.3 Memorandum of Understanding (MOU)
MOUs are broad, general agreements between NSF and other parties to pursue activities of
mutual interest and benefit; cooperate in areas where science and engineering interests
coincide; and provide a framework for cooperation. A typical MOU includes:





The purpose of the Understanding; authority of the parties to enter into an
Understanding;
Scope of the Understanding, including a project description and the respective
responsibilities of each party for funding, management and oversight (including
procedures for resolving conflicts and dealing with defaults);
Rights of each party with respect to access, ownership and intellectual property
(Chapter VII of the PAM); means for resolving disputes; and
A termination clause.

MOUs are developed by the PO and cleared according to procedures outlined in Chapter VIII of
the Proposal and Award Manual (PAM).

Section Revision:
June 17, 2015

4.6.3-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.1 Introduction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5
5.1

SPECIAL TOPICS AND SUPPLEMENTARY MATERIALS
INTRODUCTION

This section contains extensive supplementary information on special topics having to do with
the National Science Foundation (NSF) role in planning, oversight, and assurance of large facility
projects. The materials are presented in a tutorial format to be of particular benefit to
individuals newly involved with large facility projects. They are based primary on current
standards and best practices for project management.

Section Revision:
July 17, 2015

5.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.1 Introduction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2

RISK MANAGEMENT GUIDELINES

5.2.1 Introduction
Project risk management is a process which increases the probability of a successful project by
identifying threats to the project, assessing the nature of those threats, and identifying actions
that can be taken to either reduce the probability of those threats occurring or reduce the
impact of the threats to the project. Even on a simple project, things seldom go as planned.
With the highly-technical, scientifically ground breaking, and long duration projects undertaken
by the NSF there will be many changes required to the baseline plan as a project matures.
Successful projects anticipate problems, work to avoid those problems, and limit the impact
those problems will have on a project.
Risk management serves two purposes; one is to forecast impacts of possible events on the
project’s cost and schedule, the other is to prioritize and inform project decisions on alternate
strategies to mitigate the cost or schedule impact of a possible event or increase the technical
performance margin of a system or subsystem. The former (quantitative risk analysis) creates a
framework for quantifying the risks to the project goals in terms of cost in dollars, schedule in
days, and performance for the purpose of forecasting the final cost, schedule, and performance
of the complete project. The latter (qualitative risk analysis) helps the team sort through the
hundreds and perhaps thousands of risk to identify and address the ones that are most likely to
have the most significant impact on the project.
Qualitative risk analysis practices have remained relatively unchanged recently while
quantitative risk analyses have been evolving rapidly as the software tools and their integration
with scheduling software packages have evolved. While quantitative risk analysis has become
easier and more sophisticated, it is unlikely to fully replace qualitative risk analysis because the
quantitative analysis requires validated inputs that are more labor intensive to produce. Most
projects utilize the qualitative risk analysis practices for their month-to-month risk management
and implement quantitative risk analysis only when they need to re-forecast the estimate at
complete cost and completion date of the project.
Risk management involves all project personnel. With an effective risk management project
every project team member should be able to state the top project risks as well as the top risks
to their subsystem. Risk management has an inherent Malmquist (completeness) bias – there
will always be more risks to a project then are reflected in the risk register. To minimize this
effect every project team member from every perspective in the program should be
contributing threats, opportunities, and mitigation ideas to the risk board. The team also needs
to be well aware of the risks associated with their subsystems so they recognize how a mistake
in their area would impact the overall project (an aspect of human error prevention and project
safety).

Section Revision:
May 2, 2016

5.2.1-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.1 Introduction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Some projects refer to risk management as risk and opportunity management, to emphasis to
the team that they should also be thinking about opportunities for changes in the baseline plan
that could save cost, save schedule, or improve performance. This section follows the Hulett
definition of risk that is in the Project Management Body of Knowledge (PMBOK) that includes
opportunities in the definition of risk. Project teams should remind each other to keep thinking
about new opportunities as well as threats to the project.
Risk and opportunity management feeds into the key decisions that make a project successful.
It is a core activity for project managers, systems engineers, subsystem leads, program officers,
and review panels.
NSF requires large facility recipients to develop and follow formalized Risk Management during
the design and construction stages of sponsored Major Research Equipment and Facilities
Construction (MREFC) projects.
Successful Risk Management entails early recognition, proactive planning, and aggressive
execution of all risk management processes. Ideally Risk Management begins as early as the
initiation stage of the project life cycle. This guide provides detailed information on the Risk
Management1 methodologies and strategies commonly applied during project planning and
execution.
There are three key products of Risk Management as applied to NSF construction projects:




A Risk Management Plan that sets out how risks will be identified and managed by the
project following standard risk management processes and practices,
A Risk Register, or tracking tool, that documents identified risks, and
A determination of Risk Exposure and the related amount of Contingency needed to
control risks, based on quantitative risk analysis.

The Risk Management Plan (RMP) is a required element of the Project Execution Plan (PEP)
described in Section 3 of this manual (often as a separate document). A RMP should be
included in the project planning and proposals no later than the start of the conceptual design
phase. The Plan should identify the responsibilities for risk management and describe the Risk
Management process that will be followed— including roles and responsibilities, procedures,
criteria, tools, and techniques to be used to identify, analyze, respond to, and track project
risks. The level of detail in the plan, and the scope, timing, and level of risk analysis should be
commensurate with the maturity and complexity of the project and may evolve and change
over time. An example of an acceptable RMP outline is shown in Table 5.2.5-1.

1

The NSF Program Officer, as part of oversight responsibilities, identifies project-related agency risks to NSF, formulates
mitigation strategies, and documents them in an Internal Management Plan (IMP), accessible only to NSF staff.

Section Revision:
May 2, 2016

5.2.1-2

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.1 Introduction
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

The Risk Register – typically a spreadsheet or data base – is a tracking tool that includes a
description of all risks that are deemed to be important to achieving project success, along with
an assessment of those risks that allows them to be prioritized for effective management. The
Risk Register also includes the risk handling strategy, the person to whom each risk has been
assigned for accountability purposes, the current status of the risk handling strategy, and
comments. An example of a commonly used style for a Risk Register is given in Table 5.2.5-1. It
should be noted that appropriate tracking tools will vary among projects because the types of
information and indicators being monitored vary from project to project. The selection and
definition of a tracking system to be used in a project should be commensurate with the size
and complexity of the project and should be defined in the project’s RMP.
Risk Management strategy involves the estimation of overall risk exposure and the
determination of an adequate amount of contingency – a quantity of money, scheduled time,
or reductions in scope intended to recover project objectives if uncertainties and risks occur
with negative impacts. The values for cost and schedule contingencies are taken from
distributions generated by Monte Carlo simulations with probability and impact ranges for
uncertainty and risks for activities defined in the baseline. The confidence levels for meeting a
chosen project end date and total cost should lie between 70% and 90%. Scope contingency
involves identifying lower priority tasks that can be delayed or dropped from the project
without a crippling impact to project objectives. De-scoping may be used if the project forecast
indicates that cost or schedule overruns are likely. For NSF MREFC projects, these contingencies
are held separately from the project Performance Measurement Baseline (PMB) cost, schedule,
and scope. Strategies for using contingency are detailed in the project Risk Management Plan.
Contingency is controlled and managed through the project Configuration/Change Control
Process (CCP). The use of contingency is subject to approval by project leadership, and by NSF,
if amounts are above certain thresholds, as defined in the cooperative agreement (CA).
While the text of this section tends to refer to projects in construction, good risk management
practices can be useful throughout a project’s life cycle, including during operations. “The best
laid schemes of mice and men / Often go awry.” Implementing preventative mitigations and
pre-planning alternative strategies will reduce the likelihood and impact of these events.
The following subsections provide guidelines for planning the Risk Management processes,
developing the RMP, creating a Risk Register, and calculating a quantized measure of risk
exposure that leads to the establishment of contingencies. Examples of accepted or good
practices are included as guidelines.

Section Revision:
May 2, 2016

5.2.1-3

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.2 Definition of Project Risk and Risk Exposure
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.2 Definition of Project Risk and Risk Exposure
Risks are defined many ways. One of the most inclusive definitions, and the one used in these
guidelines, is; “… an uncertain event or condition that, if it occurs, has a positive or negative
effect on at least one project objective.”1
Most international standards agree that risk is made up of both threats and opportunities.
Capturing and capitalizing on opportunities to reduce costs, save time, and improve technical
performance may improve the possibility of finishing on time, budget, scope and quality by
offsetting the negative impact of threats to those objectives. The tools and methods employed
in managing threats are also used to identify and take advantage of these opportunities for
reducing project cost or schedule or improving technical performance. NSF requires
Opportunity Management as a necessary component of risk management.
Project Risk Exposure is the quantized result on project objectives of various risks and
uncertainties occurring. Project risk exposure is usually expressed as an amount of budget or
time that is the output of a Quantitative Risk Analysis that combines probability of occurrence
with consequence. Project risk exposure diminishes over time as risks are realized or avoided
and should always be less than or equal to remaining contingency amounts.

1

This definition is used in the Guide to the Project Management Body of Knowledge, (PMBOK® Guide), Project Management
Institute, 5th Edition, 2013, Chapter 11.

Section Revision:
May 2, 2016

5.2.2-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.3 Definition of Allowable Contingencies
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.3 Definition of Allowable Contingencies
Contingencies are a necessary component of risk management for NSF projects – they provide
the wherewithal and flexibility to control risks and realize opportunities. Contingency
allocations are for in-scope deliverables and are to be used to mitigate identified risks and
uncertainties that may impact a projects ability to achieve approved project objectives.
5.2.3.1 Allowable Contingency
Most risk management guides define two general types of budget and schedule contingency:
Contingency: “a planned amount of money or time which is added to a baseline
estimate to address specific, identified risks.”1
Management reserve: “a planned amount of money or time which is added to a
baseline estimate to address unforeseeable events.”2
NSF does not carry management reserves as defined above. For NSF projects, only the first
type, Contingency, is allowed. This means that the estimation of contingency amounts must be
tied to risks identified at the time the total budget and duration are set, and that such
contingency can only be used to mitigate those pre-identified risks. Use of contingency to cover
unforeseen events is not allowed. See Section 5.2.7 for using proper quantitative estimating
methodologies for determining risk based contingencies.
In addition to budget and schedule contingency planning, NSF requires projects to assess
possible use of scope contingency and to develop a plan to make effective use of scope
contingency options, if necessary, during construction. This provides the project with an
additional tool to manage the overall project given the lack of Management Reserve within
NSF. Use of all contingency is managed through formal change control processes, as described
in Section 4.2.5.
5.2.3.2 Contingency Definitions
Contingency for NSF projects includes cost, schedule, and scope amounts, as defined below:
Budget Contingency: An amount added to a baseline budget estimate to allow for identified
items, conditions, or events for which the state, occurrence, or effect is uncertain and that

1

Identified risks are often referred to as “known unknowns” in the literature. In other words, a risk that can be identified during
planning is “known,” but the probability of occurrence and the extent of its impact cannot be determined with accuracy and are
therefore “unknown.”

2

Unforeseeable events are those that are not or cannot be identified during planning and are typically referred to as “unknown
unknowns” in the literature. They may also include low probability, extreme events that are beyond project control, such as the
effects of terrorism and war, natural disasters with impacts beyond expected historical ranges, or global economic crises.

Section Revision:
May 2, 2016

5.2.3-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.3 Definition of Allowable Contingencies
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

experience shows will likely result, in aggregate, in additional costs. Typically estimated
using statistical analysis or judgment based on past asset or project experience.
For MREFC construction projects, the amount of budget contingency is determined by
performing a probabilistic risk analysis on the baseline cost and schedule and selecting a
Total Project Cost with an acceptable confidence level (typically between 70%-90%). See
Section 4.2.5 for details on Total Project Cost requirements and caps. Thus Total Project
Cost is the sum of the baseline budget and the selected contingency amount. Budget
contingency is held separately from the PMB and allocations of budget contingency to and
from the PMB are managed through formal change control.
Schedule contingency: An amount added to a baseline schedule estimate to allow for
identified delays, conditions, or events for which the state, occurrence, or effect is uncertain
and that experience shows will likely result, in aggregate, in additional project duration.
Typically estimated using statistical analysis or judgment based on past asset or project
experience.
For MREFC construction projects, the amount of schedule contingency is determined by
performing a probabilistic risk analysis on the baseline schedule of activities and selecting a
commitment finish date with a confidence level between 70%-90%. The overall project
duration is the sum of the baseline duration and the selected contingency amount.
Schedule contingency is held separately from the PMB and allocations of schedule
contingency to and from the PMB are managed through formal change control.
Scope contingency: Scope included in the project baseline definition that can be removed
without affecting the overall project’s objectives, but that may still have undesirable effects
on facility performance. Identified scope contingency should have a value equal to at least
10% of the baseline budget.
Scope contingency can be retained or deleted, depending on project risk performance and
available contingency, in order to stay within the Total Project Cost. A scope contingency
plan includes a time-phased estimate of available budget and or time from de-scoping
options, based on key decision points. See Section 4.2.5 for details on requirements.
Implementation of scope contingency options is managed through formal change control.

Section Revision:
May 2, 2016

5.2.3-2

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.4 Risk Management Steps and Methodology
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.4 Risk Management Steps and Methodology
The steps involved in the Risk Management process have been defined variously by different
practitioners. For the purposes of these guidelines, the Risk Management process is defined as
comprising the following steps:1







Risk Management Planning
Risk Identification
Qualitative Risk Analysis
Quantitative Risk Analysis
Risk Response Planning
Risk Monitoring and Control

The relationship between these steps is shown in Figure 5.2.4-1.
Figure 5.2.4-1

Picture of Six Risk Management Processes (According to PMI)2

Risk Management
Planning

Risk
Identification

Risk Monitoring
and Control

Qualitative Risk
Analysis

Risk Response
Planning

Quantitative Risk
Analysis

The Risk Management steps outlined above are iterative and continuous and any one step, or
all of the steps, could be active at any given time. Risk analysis is performed continuously
1

The six steps are the same as the processes described in the PMBOK® Guide Chapter 11.

2

Risk Handling and the Project Management Institute’s Risk Response Planning process are somewhat equivalent. Risk Handling
includes implementing the risk mitigation and other responses, whereas Risk Response Planning envisions that these actions
will be included in the Project Execution Plan and implemented as part of the PEP.

Section Revision:
May 2, 2016

5.2.4-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.4 Risk Management Steps and Methodology
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

throughout the project life cycle. For example, a conceptual risk analysis may be conducted to
facilitate selection between alternative options, to determine the level of project management
required, to identify where the challenges lie, and to determine the level of technical
information and development activity necessary to achieve project success. That risk analysis is
then updated during each of the life cycle phases of the project. Performing risk analysis is
particularly necessary in preparing for key project decisions. Periodic reviews of the risks at
appropriate intervals should be performed to identify new risks, to evaluate progress in risk
handling strategies, as well as to evaluate changes during the project development and
implementation cycles. Risk Management Planning, and the RMP, may also need to be
re-addressed at times of significant change, such as transitions from one project life cycle to
another or during a re-baselining with significant modifications to the project baseline.

Section Revision:
May 2, 2016

5.2.4-2

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.5 Risk Management Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.5 Risk Management Planning
Planning begins by developing and documenting a Risk Management strategy. Early efforts
establish the purpose and objectives, assign responsibilities for specific areas, identify
additional technical expertise needed, describe the assessment process and areas to consider,
delineate considerations for mitigation planning, define a rating scheme, dictate the reporting
and documentation needs, and establish report requirements. The strategy to manage root
causes provides the program team with direction and a basis for planning. The output of risk
management planning is a written document – the Risk Management Plan (RMP) – containing
the details of how risk will be managed through application of tools and processes defined in
the plan. See the next subsection for a description of requirements for the RMP.
One key strategic decision that should be made early in the Risk Management planning is the
selection and assignment of personnel with appropriate capability in Risk Management to lead
and/or guide the planning and analyses. As will be seen from the topics presented in the
analysis portion of the section, the art and science of risk management can be extremely
complicated for complex, high risk projects. While project managers, scientists and engineers
may have expert knowledge and judgment for identifying, estimating impacts from, and
defining mitigation for individual risks, they are usually not expert in estimating the overall or
aggregate risk exposure to the project from the combined impact of many individual risks.
Finding qualified resources to meet the risk management requirements of the project,
particularly for establishing the amount of contingency, should be a high priority for early
planning in order to ensure that methodologies and programming tools can be selected and
implemented in a timely manner. Options include sending existing staff for specialized training
in risk management and tool usage, directly hiring risk management experts, contracting with
industry, or some combination of the above.
A second early key decision is the determination of what risk assessment methodologies and
tools will be used, from first estimates through construction. The sophistication of the
appropriate risk assessment tools typically increases with advancing planning detail and
maturity, as well as with increasing project complexity. A project that includes a high number of
procurements and in-house tasks typically requires software applications and methods that use
a fully resource loaded schedule for risk assessment and contingency estimation, while a
project entailing management of a single large contract may be adequately served by tools and
methods that use cost spreadsheets and summary level schedules. Choosing the appropriate
tools and methods at the outset can avoid the need and the burden of changing to different
systems as the project planning matures.
Risk Management planning is iterative. Normally, the risk management methodology and
procedures are defined as part of the risk management process planning early in the design
stage, but they may be extended or modified during design and execution as long as the efforts
remain within approved scope.

Section Revision:
May 2, 2016

5.2.5-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.5 Risk Management Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.5.1 Risk Management Plan (RMP)
The Risk Management Plan (RMP) describes how risk management will be applied on the
project. It is an integral part of the PEP, as outlined in Section 3.4 of this manual. The level of
detail in the plan and the scope, timing, and level of risk analysis should be commensurate with
the complexity and maturity of the project as it advances through design and construction
stages. The plan is a living document used throughout design and implementation and should
therefore be under configuration management. The Risk Management Plan should include the
following elements:







Risk Management Strategy and Approach
Roles and Responsibilities
Processes used to apply the Risk Management process
Baseline definition for Calculating Risk Exposure and Contingency needs
Contingency Estimating and Management
Resources assigned to and schedule, cost, and timing of risk management activities

The Recipient should periodically review the RMP and revise it, if necessary. Some events may
drive the need to update an existing RMP, such as: (1) the baselining of a project,
(2) preparation for a major decision point, (3) technical audits and reviews, (4) an update of
other project plans, and (5) a change in major project assumptions. A sample format with the
expected content for a Risk Management Plan (RMP) is outlined in Table 5.2.5-1.
Table 5.2.5-1

Sample Format for a Risk Management Plan

Section

Description

1.

Introduction

This section should address the purpose and objective of the plan, and provide a
brief summary of the project, to include the approach being used to manage the
project, and the acquisition strategy.

2.

Definitions

Definitions used by the Recipient should be consistent with NSF definitions for ease
of understanding and consistency. However, the NSF definitions allow program
officers flexibility in constructing their risk management programs. Therefore, each
Recipient’s RMP may include definitions that expand the NSF definitions to fit its
particular needs. For example, each plan should include, among other things,
definitions for the ratings used for technical, schedule, and cost risk in qualitative
risk analysis.

3.

Risk
Management
Strategy and
Approach

Provide an overview of the risk management approach, to include the status of the
risk management effort to date, and a description of the project risk management
strategy.

4.

Organization

Describe the risk management organization of the Recipient and list the roles and
responsibilities of each of the risk management participants.

Section Revision:
May 2, 2016

5.2.5-2

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.5 Risk Management Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Section

Description

5.

Resources
Implications of
the Plan

The resources to be used in managing risk on the project should include the time of
management and project team members as well as risk specialists and contractors if
appropriate, to manage effectively the risks on the project. These risk management
costs should appear specifically in the project budget.

6.

Schedule
Implications of
the Plan

The time periods in the project schedule when risk management activities are
planned to occur. Activities providing sufficient time to perform the tasks and
milestones to record their completion should be inserted in the project schedule
and statused along with the schedule statusing plan.

7.

Risk
Management
Process and
Procedures

Describe the project risk management process to be employed, i.e., risk planning,
qualitative and quantitative risk assessment, handling, monitoring and
documentation, and a basic explanation of these components. Also provide
application guidance for each of the risk management functions in the process. If
possible, the guidance should be as general as possible to allow the project’s risk
management organization flexibility in managing the project risk, yet specific
enough to ensure a common and coordinated approach to risk management. It
should address how the information associated with each element of the risk
management process will be documented and communicated to all participants in
the process, and how risks will be tracked to include the identification of specific
metrics if possible.

8.

Risk Planning

This section describes the relationship between continuous risk planning and this
RMP. Guidance on updates of the RMP and the approval process to be followed
should be included.

9.

Risk
Identification

This section of the plan describes the identification process. It includes procedures
to be used for examining the critical risk areas and processes to identify and
document the associated risks.

10. Risk Register
Analysis and
Ranking

This section summarizes the analyses process for developing a qualitative or
quantitative risk rating and populating the Risk Register. This rating is a reflection of
the potential probability of each risk and the impact of each risk on the project
schedule, cost, scope and quality. It also describes how the risk analysis data will be
collected and maintained throughout the project’s life cycle.

11. Probabilistic Risk
Analysis and
Contingency

This section describes the way the project will analyze the implications of identified
and quantified risks on the total project schedule and cost objectives or major
milestones. Typically a Monte Carlo simulation is used based on the project
resource-loaded schedule or on the cost estimate if a schedule is not available. This
section also describes the use of the risk analysis results for setting contingency
amounts and prioritizing risks for risk mitigation.

12. Risk Handling

This section describes the risk handling options, and identifies tools that can assist
in implementing the risk handling process. It also provides guidance on the use of
the various handling options for specific risks.

13. Risk Monitoring

This section describes the process and procedures that will be followed to monitor
the status of the various risk events identified including the frequency and
organizational level of risk review. It provides for identification and calibration of
new risks should they arise. It should provide criteria for the selection of risks and
risk mitigations to be reported on, and the frequency of reporting. Guidance on the
selection of metrics should also be included.

Section Revision:
May 2, 2016

5.2.5-3

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.5 Risk Management Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Section

Description

14. Risk
Management
Information
System,
Documentation
and Reports

This section describes the management information system structure, rules, and
procedures that will be used to document the results of the risk management
process. It also identifies the risk management documentation and reports that will
be prepared; specifies the format and frequency of the reports; and assigns
responsibility for their preparation and dissemination.

15. Risk Exposure
and Contingency
Management

This section describes the specific process and procedures used to determine
construction project risk exposures and the concomitant contingencies for scope,
cost, and schedule. It describes contingency management plans and processes and
ensures that contingency use is linked to both an identified risk and an appropriate
Work Breakdown Structure (WBS) element within project scope.

5.2.5.2 Roles and Responsibilities
Typically, the Project Manager or a designated Risk Manager (RM) is responsible for leading the
identification and analysis of project risks. All stakeholders (e.g., users, designers, and sponsors)
involved in the project are asked to provide input on what they deem to be the risks for the
project, possible risk mitigations, and ways to capture potential opportunities. The RM
consolidates the information collected and creates the list of risks with accompanying
attributes, and manages the response to the risks. An example of a Roles and Responsibilities
table for key stakeholders and project staff that meets requirements is shown below in Table
5.2.5-2.
Table 5.2.5-2
Roles

Example of a Risk Management Roles and Responsibilities Table
Responsibilities

Organization
Management

 Support the risk management process. Encourage all levels of the project organization to
participate fully and openly in the process.
 Make project decisions based in part on the results of risk analysis.
 Provide the culture that supports risk management and welcomes honest and realistic
results.

Risk Manager

 Oversee the Identification and documentation of new risks (threats and opportunities)
in the risk register
 Oversee the analysis of risks by the project team and work with them to develop risk
response plans (mitigate, avoid, accept, and transfer).
 Oversee reporting and tracking of risk activities during project status meetings
 Document and communicate risk activities frequently with stakeholders
 Review risks as they are concluded, and identify lessons learned
 Recommend and champion mitigation strategies to the Change control Board (CCB) on
behalf of the risk management team.

Section Revision:
May 2, 2016

5.2.5-4

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.5 Risk Management Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Roles

Responsibilities

Project Team
Members

 Assist the RM with the risk identification, qualitative and quantitative risk analysis and
development of risk response plans (mitigate, avoid, accept, and transfer). Participate in
risk workshops and interviews to provide risk data.
 Submit new threats, opportunities, and mitigations into the risk system as they arise.
 Assist the RM with the development and execution of risk response plans
 Attend project risk status meetings, as needed, and assist RM with the reporting and
tracking of risk activities
 Assist the RM with documenting and communicating the risk (threats and opportunities)
activities frequently with stakeholders
 Review risks as they are concluded, and identify lessons learned

Risk Owner

 Assist the risk originator (PM, RM, project team member, etc.) with development of the
risk descriptions
 Assist the RM and project team with the analysis development of risk response plans
(mitigate, avoid, accept, and transfer) contingency plans
 Update the risk register with modifications to risks
 Monitor the risk triggers and update the risk register
 Attend project status meetings, as needed, and assist the PM with reporting and
tracking risk activities
 Assist the RM and project team with documenting and communicating risk activities
with stakeholders
 Capture risk closure notes in the risk register and lessons learned

Section Revision:
May 2, 2016

5.2.5-5

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.6 Risk Identification
5.2.6.1 Risk Identification Process
Risk identification is an organized approach for determining, listing, and describing events that
might impact a project’s objectives (for example, time, cost, scope, and performance). It is an
iterative process that is conducted throughout the entire project life cycle. The risks are
described with a basis as to why this event is considered a risk. Identification relies on the skill,
experience, and insight of project personnel and subject matter experts (SMEs), as well as the
Recipient’s project manager, the NSF Program Officer, and the NSF Grants and Agreements
Officer. The objective of risk identification is to describe all the relevant risks so that the group
can focus on uncovering the probability and impact of the risks on project objectives (used in
qualitative risk analysis) or activities / costs affected (used in quantitative risk analysis). The
process for performing Risk Identification, along with inputs and outputs, is given in Figure
5.2.6-1.
Figure 5.2.6-1

Risk Identification Process

Process
• Project schedule,
scope, budget
• System and project
artifacts
• Meeting Minutes
• Environmental
factors, PEP

Inputs

• Review key project/ phase deliverables
• Meet with the Risk Manager and project team
on a regular basis to discuss and identify
potential risks
• Interview key stakeholders
• Review existing risks to validate
• Update risk or add new risk to Risk Register
• Brainstorm for risk at status meetings
• WHEN: at the beginning of projects, project
milestones, and at weekly/ monthly status
meetings
•

• New and updated
risks
• Updated Risk
Register with
description
• Potential risk
responses

Outputs

Techniques used in identifying risks include leveraging existing project artifacts and guidance
documents, as well as proactively searching and gathering information to assist in that identification. The quality and completeness of risk identification is primarily dependent upon the
knowledge and experience of the project team and its commitment to risk management
processes. For example, the following basic methods can be used to assist in the identification
of risks:

Section Revision:
May 2, 2016

5.2.6-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)







Brainstorming
Diagramming
Interviewing
Analysis of existing project artifacts
Comparison to historical information

Formal risk identification is performed in the early part of the project life cycle and as part of a
continuous effort during the project life cycle. Any person associated with the project should be
encouraged to continually identify potential project risks. Risk Identification, whether in
workshops or in interviews, should include at least the following participants:






Project managers
Project team leaders
Project team members
Business stakeholders
SMEs





Contractors
SMEs outside the project team, for unbiased
perspective
Project partners (e.g., foreign agencies,
organizations with diverse objectives)

One immediate outcome of risk identification is the populating of the Risk Register, or tracking
tool, with the identified risks. The priorities based on impacts to project objectives and plans for
handling and reducing impacts will be added after analysis and risk handling planning, as
described in later sections. The Risk Register provides a means of tracking and reporting status
as risks occur and migration strategies are implemented, and is an important tool for Risk
Management implementation.
5.2.6.2 Risk Identification and the Risk Register
The Risk Register includes a description of all risks that are deemed to be important to
achieving project success (from the Risk Identification process) along with an assessment of
those risks (using Qualitative Risk Analysis) that allows risks to be prioritized for effective risk
management. The results of identification, qualitative analysis, and risk handling – the major
components in the Risk Register – can lead to further analysis (Quantitative Risk Analysis, for
example).
Each risk should be assigned a unique identification number or code. Once a risk is entered in to
the Register, it is never deleted. Its state may be changed to inactive (for example, retired,
closed, or merged with another risk), but it should never be deleted from the register. Risk IDs
are never reused.
The Register should be accessible (read-access) to all project members – the primary objective
of the Register is to keep the project team thinking pro-actively about how to avoid or mitigate
threats and take advantage of opportunities. It can be a spreadsheet, data base, or a specialized

Section Revision:
May 2, 2016

5.2.6-2

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

risk management software tool. Changes to the Register should be managed through the risk
process, which often may restrict ability to make changes (write-access) to a small team or to
the Risk Manager. The mechanism for all project members to read, comment on, and submit
new risks and mitigations should be established in the RMP.
The examples shown here and in Section 5.2.7, Qualitative Risk Analysis, lead to a Risk Register
containing a ranked subset or summarized set of risks based on individual, qualitative impact
analysis. Note that numeric impacts determined by the qualitative method for individual risks
may not be summed or combined to give a value for overall project risk exposure.
Further discussion of Risk Register content is given in Section 5.2.7, Qualitative Risk Analysis,
with a sample Risk Register shown in Figure 5.2.7-6.
5.2.6.3 Risk Description
The risk description serves as a key point in the Risk Register, and will be generated and
updated as needed. If there is a trigger event that causes the risk or foretells the risk’s occurring
it should be described since it will specify what condition(s) would launch the risk and maybe
activate a contingency plan.
Risks (both threats and opportunities) are typically identified and tracked using the following
sentence structure for the Risk Description:
“Because of (some cause) a (risk) may occur, and (consequences) will happen.”
Example:
“Because foreign exchange rates may change, the cost of components in WBS 3.1 and
2.6 may increase or decrease, causing cost variances which affect contingency use.”
Using this format helps to distinguish the uncertainty or risk from its cause and its
consequence, a distinction which is important for mitigation planning. For instance, a statement
that “we have 5 schedule risks” is focused on an objective (schedule) that is impacted, not the
root cause of the risk or uncertainty. Alternatively, a statement that “the risk is that the
technology is really hard” does not lend itself to mitigation efforts. Difficulty of technology is a
fact or a “cause” in this format, which cannot be changed. The risk may be that “we do not have
the right skills on the project to handle the complexity” or “we may have to rely on third parties
to gain control of this technology.” That is a risk that can perhaps be mitigated. A possible risk
description for this scenario may be:
Because the technology for the major components in WBS 2.7 is very advanced, we may
have to rely on third parties for design, with the consequence that we have less control
over cost and schedule and an added burden of increased communication efforts.

Section Revision:
May 2, 2016

5.2.6-3

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Risks should be identified that are both internal, perhaps under project control, and external,
likely to be beyond project control. Risks for which there are no plans for mitigation should also
be included in the register.1 However, note that NSF does not allow the use of contingency for
risks that are commonly referred to as “unknown unknowns”2 such as exceptional events or
major changes in scope. These exceptional events may not be included in quantitative risk
analyses used in determining contingency amounts.
5.2.6.4 Risk Identification Concerns
Efforts should be made to identify all risks to the project as early as possible, employing all
stakeholders identified in and using the techniques listed in Section 5.2.6.1, Risk Identification
Process. At the time of the risk analysis there are likely to be risks that are currently unknown
but may be revealed at a later date. When they become apparent, they can be then analyzed
and a “corrective action” can be specified and implemented. The objective of the Risk
Management program is to minimize the number of unanticipated issues and to address them
when possible and prudent.
Some people believe that project risk is often underestimated in both cost and schedule,
leading to well-known, sometimes notorious, overruns.3, 4 Historical experience suggests that
mega-projects suffer from such problems systematically.5 Strategic or overarching risks are
often missed in the risk identification process since the participants do not think globally, only
locally. Systemic or overarching risks are often not discussed or even considered during risk
identification. There may be cultural bias that leads to optimistic thinking in which threats are
systematically underestimated, outcomes are assumed to be achievable with less than realistic
effort and the potential for set-backs and rework is ignored. Any tool or technique that will
encourage people to “think outside of the box” when identifying project risk will help identify
the possibility of large overruns – when caught early, these risks may be manageable.
One common issue in identifying and collecting project risks is that people’s response and
participation in the identification process may be “stove-piped.” That means that people will
ordinarily discuss threats and opportunities that have to do with their own area of
1

The fact that “we cannot do anything about it” or “we choose to accept the risk” does not disqualify it as a risk to the project.
One can argue that these risks must be in the Risk Register and certainly in the Quantitative Risk Analysis.
2

In many Risk Management guides, a portion of contingency is designated as management reserve for “unknown unknowns.”
NSF does not allow what is normally referred to as management reserve. The current NSF budget cap policy and scope decrease
plans of 10% of the Project Management Baseline replaces reserve.
3

See Bent Flyvbjerg, Nils Bruzelius, and Werner Rothengatter, Megaprojects and Risk: An Anatomy of Ambition, 2003
Cambridge University Press, and Glenn Butts and Kent Linton, “NASA the Joint Confidence Level Paradox – a History of Denial,”
NASA Cost Symposium, April 28, 2009.
4

Challenges to Meeting cost, Schedule, and Performance Goals, NASA IG-12-21

5

Edward W. Merrow, Industrial Megaprojects, 2011, Wiley.

Section Revision:
May 2, 2016

5.2.6-4

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

concentration. In practice the project teams and other SMEs have experience and knowledge
outside these narrow areas, so the data collection method used should encourage them to
think broadly and strategically when identifying risks. Reminding risk identification participants
of external, organizational and project management source-areas of risks can help elucidate
strategic risks that they know about but that are outside their narrow area of technical
expertise or their work assignment. Often the use of a standard Risk Breakdown Structure
shown in Figure 5.2.6-2 will encourage risk identification participants to think more broadly
about risks to the project.
Figure 5.2.6-2

Typical Risk Breakdown Structure (RBS)1

Project

Technical

External

Organizational

Project Management

Requirements

Subcontractors
and Suppliers

Project
Dependencies

Estimating

Technology

Regulatory

Resources

Planning

Complexity and
Interfaces

Market

Funding

Controlling

Performance and
Reliability

Customer

Prioritization

Communication

Quality

Weather

Different approaches to Risk Management may subdivide risks into various categories for
analysis. For illustrative purposes, this guide will use cost, schedule, and technical or
performance risk as the categories used in examples. Other risk categories commonly in use are
programmatic, business or economic, design requirements, software, and technology risks.
Alternatively, the OMB Risk Categories shown in Figure 5.2.6-3 could be used as guidelines for
identifying the various types of risk that apply to the project (refer to “OMB Risk Categories”
document in Critical Infrastructure Protection (CIP) for detailed descriptions and examples of
these categories). See the Government Accountability Office (GAO) Cost Estimating and
Assessment Guide, GAO-09-3SP, Chapter 14, for more examples. Some projects may also decide
to differentiate between internal and external risks. In many cases, it may be advisable to use
1

This RBS was the initial model for the RBS in the PMBOK® Guide, Chapter 11 of the Project Management Institute

Section Revision:
May 2, 2016

5.2.6-5

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

different categories for various parts of a project. For instance, categories of risk may be
different for software development than for hardware procurement. Each project should
decide which categories are most appropriate for its use while establishing the Risk
Management Plan and processes.
Figure 5.2.6-3

1)
2)
3)
4)
5)
6)
7)
8)
9)
10)

OMB Risk Categories: to be used as a starting point for projects to select their own categories

Schedule
Initial Costs
Life Cycle Costs
Technical Obsolescence
Feasibility
Reliability of Systems
Dependencies and Interoperability
Surety (Asset Protection)
Risk of Creating a Monopoly
Capability of Agency to Manage the
Investment

11) Overall Risk of Project Failure
12) Organizational and Change
Management
13) Business
14) Data/Info
15) Technology
16) Strategic
17) Security
18) Privacy
19) Project Resources

Another related social or cultural issue in identifying project risks is that people are often
uneasy about (or even afraid to be) discussing risks that can be embarrassing or harmful to the
project. This unease is often experienced during risk workshops or in other group settings.
Social pressures to conform (“groupthink”) – to suppress dissenting opinions clearly unpopular
to the group, including management, to agree with others against personal opinion just to
move the workshop along, and to defer to people perceived to have greater expertise even
when in disagreement – often make it difficult for some people to speak out.1 A possible
solution to the impacts of social pressure is to conduct one-on-one, in-depth interviews with
SMEs in which the interviewee is promised confidentiality. Such interviews often yield honest
opinions about what might affect the project’s success. Usually some or most of the risks
revealed and discussed in these sessions are not on the organization’s risk register and would
not be analyzed in qualitative or quantitative risk analysis in the absence of the interviews. For
these reasons it is important to provide a safe environment for project team members and
others to identify and discuss project risks.
Risk identifiers may have concerns about including risks that are 100% likely to happen
(sometimes these are called “uncertainties or issues”) in the Risk Register. If the risks are 100%
likely to happen and their impacts are known, they should be included in the PMB. Often,
however, a risk that is certain to occur will have an impact that is not already included in the
project execution plan and that needs to be handled somewhere else, such as in the risk
1

These phenomena are discussed in Understanding and Managing Risk Attitude, David Hillson and Ruth Murray-Webster,
Gower, 2005.

Section Revision:
May 2, 2016

5.2.6-6

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.6 Risk Identification
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

analysis. Or the risk may have an uncertain impact on project objectives. These situations call
for the risk to be identified, even if it is already happening or certainly will happen, so the risk
can be included in the Risk Register and the subsequent quantitative risk analysis. The objective
of risk identification is to describe all the relevant risks so that the group can focus on
uncovering the probability and impact of the risks on project objectives (used in qualitative risk
analysis) or activities / costs affected (used in quantitative risk analysis). Once a risk that is
certain to occur has been included in the PMB it can be removed from the Risk Register.
Care must be taken to provide the same thoroughness of identification for events far in the
future as well as for the near term for projects that have an execution period of several years.
Project team members usually find that it is easier to identify and discuss the risks that are
current or on the near horizon than those that may occur much later in the project. Adding to
the difficulty is the fact that future events may not be well defined at the time of risk analysis.
The risk identification exercise should take special care to encourage the participants to look
into the future, maybe with the help of lessons learned documents or their own experiences on
prior projects, to see what risks are far down the project life cycle. Another useful technique is
to “walk through” the activities planned for later execution. Examples of unidentified risks may
include unexpected legal changes, technical performance issues, resource losses, etc. Other
sources of future risks might include the reliance on unproven or even just conceptualized
technology or on doing business with an organization or in a country unknown to the sponsors.
The risk identification should include thinking about risks that have happened on other similar
projects or might occur in later phases of this project. If the team spends some time discussing
these down-stream risks they can perhaps remember other projects or conceptualize the
existence of risks that would otherwise go unreported.

Section Revision:
May 2, 2016

5.2.6-7

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.7 Qualitative Risk Analysis – Risk Register Ranking
5.2.7.1 Purpose of Qualitative Risk Analysis
Qualitative risk analysis involves determining the probability of the occurrence of a risk,
assessing the consequences of this risk on specific project objectives (time, cost, scope and
quality) if it occurs, and using the two dimensions of a risk (probability and consequence) to
identify a rank or “risk level.” This risk level represents a judgment as to the relative risk to the
project objectives and the project as a whole and is categorized as Low, Moderate, or High.
Based on the risk level, risks can be prioritized for mitigation to responses. The results of Risk
Identification, Qualitative Risk Analysis, and Risk Response Planning comprise the main
elements of the Risk Register. The process for performing Qualitative Risk Analysis, along with
inputs and outputs, is given in Figure 5.2.7-1 below.
Figure 5.2.7-1

Qualitative Risk Analysis Process

• Risk and uncertainty
descriptions and
calibration
• Project artifacts such
as schedule, estimate
• Client and team
interviews for risk
data
• Other data gathering,
lessons learned

Inputs

Process
• Assess the schedule and cost against best
practices
• Conduct in-depth confidential risk interviews for
probability, impact and activities / costs affected
• Calibrate and assign inherent uncertainty and
estimating error, providing for more uncertainty
for work performed in the future
• Run Monte Carlo simulation on the risk-assigned
project schedule or estimate
• Prioritize the risks to the project
• Mitigate the high-priority risks and create a postmitigated result

• Likelihood of
finishing on time
and budget
• Needed cost and
time contingency
amounts
• Identification of high
priority risks and
needed risk
mitigations

Outputs

Achieving risk reduction is an integral part of setting priorities, sequencing project work, and
responding to the most serious risks first. Thus, the identified risks must be prioritized.
Note that Qualitative Risk Analysis is applied only to individual risks and is not used in
estimating overall project risk exposure or in determining contingency amounts. The analysis of
impact or consequence, however, may serve as input to Quantitative Analysis used to estimate
overall risk exposure.
5.2.7.2 Considerations When Performing Qualitative Risk Analysis
A number of factors complicate qualitative risk analysis, including:


Risk data, like data about the future contained in cost estimates or project schedules,
have a significant content based on subjective, expert judgment. The evaluation of risks

Section Revision:
May 2, 2016

5.2.7-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

to cost and schedule therefore generate approximate rather than precise results. “There
are no facts about the future.”1


The term “risk” includes both “threats” and “opportunities” for NSF purposes. Project
risk team members should look for uncertainties that could help improve the project’s
results or offset threats. Both threats and opportunities should be examined for total
project impact, since opportunities for one project participant may be considered
threats by another, and vice versa.



The probability that a risk may occur and the impact if the risk were to occur should be
evaluated separately before combining the two parameters in a risk matrix. The idea
that “the risk is unlikely so its impact must be low” confuses the two parameters of
probability and impact. SMEs should be asked to estimate the impacts as if the risk has
occurred. Probability and impact will be considered together in the combined risk matrix
approach.



It is good practice to assess risks’ impact on separate objectives such as time, cost,
scope or quality/performance Impact ranges rather than creating a single, overall risk
level for the risk. Thus, ranking levels are defined for each of these objectives. For
instance, a risk can be judged to have a high impact on time but a moderate impact on
cost and a low impact on scope.



The definitions of impact on each project objective (very low, low, moderate, high and
very high) are set by the Risk Manager and documented in the RMP.



The definitions of combined risk level for probability and impact taken together (low,
moderate or high; or green, yellow or red) in the Risk Matrix are set by the Risk Manager
and documented in the RMP.



The impact of an individual risk may be modest and still be considered a high or very
high priority for mitigation. This is because the combined or aggregate risk of many
moderate risks may be high. The project may want to mitigate some low or moderate
risk in order to reduce the combined threat from many risks.



The risk register should include only root cause risks. Risks as defined in the plan may
not be mutually exclusive, that is, they may have the same root cause risk. Put another
way, if two or more risks are not mutually exclusive as written, their common root cause
risk should be identified and used instead.

1

Lincoln E. Moses, Administrator of the Energy Information Administration, Administrator’s Message to the Annual Report To
Congress, 1977, Volume Three.

Section Revision:
May 2, 2016

5.2.7-2

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.7.3 Limitations of Qualitative Analysis
There are some limitations to the practice of qualitative risk analysis. Recognizing these will
help the organization appreciate and use the results correctly:


Qualitative Risk Analysis addresses the impact of individual risks on project objectives
one at a time. As such it is dedicated to prioritizing individual risks based on subjective
estimates of probability and subjective estimates of the impact to the project objectives.
It is not equipped to forecast or estimate overall project results such as the finish date
or total cost.



Qualitative Risk Analysis is unlikely to yield valid quantitative results since it usually does
not include all possible correlations and outcomes for impacted activities from a single
risk. SMEs must consider the risk with its probability of occurring and all the activities in
the schedule it would affect if it occurred, whether or not those activities are on the risk
critical path (the risk equivalent to the critical path in CPM scheduling). They must also
evaluate whether other risks might keep that risk from creating much improvement if it
were mitigated. All of this analysis is being done in the individual SME’s head. Such
complex calculations are best handled by the Quantitative Risk Analysis simulation
method described in Section 5.2.7.



The estimation of the impact of a risk on cost must consider the impact of that risk
directly on cost plus the impact indirectly on the cost of time-dependent resources if the
risk also affects time. This is another calculation that individuals are not well-equipped
to make without a computer but is handled well by Quantitative Risk Analysis.



Judging whether a risk has a high-priority for the project would involve reviewing the
conclusions on each objective and asking the risk manager to prioritize the objectives.
Some projects are time sensitive and some are budget driven, others have a fixed scope
or could be de-scoped. These factors would need to be considered to determine
whether the risk is low, moderate or high priority for the project as a whole. Some
software packages that perform qualitative risk analysis assume that if a risk is “high”
for any objective it should be judged to be “high” for the entire project. There is no real
basis for doing so, since the risk may be judged to be high for an objective that is not the
most important for the specific project under consideration.

5.2.7.4 Qualitative Risk Analysis – Probability and Impact Assessment
Risk level determination can be done using a variety of techniques. The method given here
begins by assigning qualitative values for event probability and impact/consequence(s) to each
objective separately. These will then be used to determine a qualitative risk level. A key feature
of this method is that it requires independent assessment of the probability and consequence
of a risk.

Section Revision:
May 2, 2016

5.2.7-3

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

The probability of a risk occurring is usually given to a range of possible probabilities of
occurring. Similarly, consequences are usually expressed in levels that represent ranges of
impacts judged by the risk manager to be of very low, low, moderate, high or very high impact
as the result of one risk among many. The ranking of a risk as it is applied to a particular
objective (e.g., time) is determined by the combination of probability and impact ranges, where
the project manager or some other stakeholder (e.g., the customer) determines which
combinations would indicate that the risk is high, moderate of low priority for further study,
quantitative risk analysis or handling.
The following steps provide the details of this Qualitative Analysis method:
1. Address each risk statement from the Risk Identification process individually.
2. Determine the qualitative probability of occurrence value (P) range that best describes
the probability for each risk with appropriate basis and justification. Discuss the
probability that the risk might occur on the project with some noticeable effect on the
objective being discussed. Estimate the probability for the risk without regard to which
objective(s) the risk affects if it occurs. The probability of occurrence is for the duration
of all project phases. Table 5.2.7-1 provides an example of typical criteria for
establishing probability values.
3. Determine the qualitative consequence or impact of occurrence value I range that best
describes the impact for the objective such as time, cost, scope or performance for each
risk with appropriate basis and justification. In the evaluation, assume that the risk has
occurred and determine the recovery time, the cost of recovery, and the impact on
scope or quality. The consequence of occurrence is for the duration of all project phases
and for the objective being assessed. Table 5.2.7-2 provides typical criteria for
establishing consequence values. This table illustrates the different definitions that are
applied to the implications for time, cost, scope and quality. Of course these definitions
should be tailored to the project by the project manager or some other stakeholder
(e.g., the owner or customer).

Section Revision:
May 2, 2016

5.2.7-4

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 5.2.7-1

Sample Risk Probabilities Table

Each project should determine the appropriate number of levels and their definitions that match that project’s
circumstances.
Probability of
Occurrence Descriptor

Probability of Occurrence
Numerical Ranges
equivalent levels1

Very Low

<0.1

Low

>0.1 but <0.4

Will not very likely occur in the life cycle of the project
or its facilities; or the probability of occurrence is
greater than 10% but less than or equal to 40%.

Moderate

>0.4 but <0.6

Will likely with middling probability (e.g., a coin flip) to
occur sometime during the life cycle of the project or its
facilities; or the probability of occurrence is greater than
40% but less than 60%.

High

>.6 but <.8

Likely to occur with more than 60 percent chance during
the project, or the probability of occurrence is between
60% and 80%

Very High

>0.8

Will likely occur sometime during the life cycle of the
project; or the probability of occurrence is greater than
or equal to 80%.

Criteria in Words
Will not likely occur anytime in life cycle of the facilities;
or the probability of occurrence is less than equal to
10%.

Sample Risk Consequences2 Table

Table 5.2.7-2

The descriptors for the objectives of cost and time are explicitly given as numbers while those for scope and quality
are expressed in narrative descriptions.
Defined Conditions for Impact Scales of a Risk on Major Project Objectives, e.g., Time
Definition for Threats Only
Project Objective

1

Very Low

Low

Moderate

High

Very High

Cost

Insignificant Cost
Increase

<5% Cost Increase

<5 - 10% Cost
Increase

<0 - 20% Cost
Increase

>20% Cost
Increase

Time

Insignificant Time
Increase

<5% Time
Increase

<5 - 10 % Time
Increase

<0 - 20 % Time
Increase

>20% Time
Increase

Scope

Scope Decreases are
Barely Noticeable

Minor Areas of
Scope Affected

Major Areas of
Scope Affected

Scope Reduction
Unacceptable to
Sponsor

Project End Item
is Effectively
Useless

Quality

Quality Degradation
Barely Noticeable

Only Very
Demanding
Applications are
Affected

Quality
Reduction
Requires
Sponsor
Approval

Quality
Reduction
Unacceptable to
Sponsor

Project End Item
is Effectively
Useless

The scales still must be calibrated per the discussion and reference in Section 5.2.9.

2

An earlier version of this table was used in the PMBOK® Guide. The percentage ranges should be adjusted by the project
manager for the project and translated into days and dollars for ease of use.

Section Revision:
May 2, 2016

5.2.7-5

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Notice that the definitions for time and cost can be quantitative but that those for scope and
quality are generally descriptive. Sometimes a project’s scopes can be quantified, though it may
have several dimensions. Quality or performance might be measurable in terms of failure rates
or number of “fixes” that would be needed. The more the impact levels can be quantified the
more the responses by different people will be comparable. The project manager can calibrate
the numerical ranges for the specific project. The consequence definitions of very low, low,
moderate, high or very high (or Negligible, Marginal, Significant, Critical, and Crisis) may vary
considerably from a small to a large project. These tables should be provided as part of the
RMP.
It is preferable to refer to the numerical levels when gathering qualitative risk data since
definitions in words are often misleading. For instance, two people may use the term “Likely”
but mean different values. Or, one may say the risk is “likely” to occur and another may say
“unlikely” but mean the same numerical value, or at least in the same “bucket” or range of
values. Research has shown that the overlap in probability values with common word
definitions is severe.1 (See Figure 5.2.7-2.)
Figure 5.2.7-2

Overlap in Risk Probability of Occurring When Descriptors Are Used

Expert judgment is required in risk analysis, just as it is for project scheduling and project cost
estimating. That is why several or many people need to be involved in providing their opinions
and experiences when assessing project risks. With multiple people assessing the probability
and impact of each risk against each objective, such as in the recommended in-depth
1

Private research conducted by Dr. David Hillson in 2004 and presented at a PMI EMEA conference.

Section Revision:
May 2, 2016

5.2.7-6

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

confidential interviews, there will be differences of opinion between them. The risk analyst has
to consolidate the data from different sources into one set of parameters for each risk and
objective from the dissimilar responses. This process uses expert judgment.
5.2.7.5 Alternative Approach to Qualitative Risk Impact Analysis ‒ Maxwell
An alternative approach is that proposed by F. D. Maxwell for projects such as those supported
by the Aerospace Corporation or the Space & Missile Systems Center of the US Air Force in Los
Angeles. See Table 5.2.7-3. The “risk driver category” is not the same as the project objectives,
but rather describes where the risks might be originating. This was not used by Maxwell in
conjunction with the probability before 1990, but it does illustrate definitions of impact that
were used on many aerospace and scientific projects. Maxwell stated that:
Special attention must be given to first-of-a-kind risks because they are often
associated with project failure. First-of-a-kind risks should receive a critical or
crisis consequence estimate unless there is a compelling argument for a lesser
consequence value determination.
Table 5.2.7-3

Maxwell Risk Driver Assessment Framework1
Risk Levels

Risk Driver Category

Very Low

Low

Medium

High

Very High

1

Required Technical
Advancement

Nothing New

Minor
Modifications
Only

Major
Modifications

State of the Art

Beyond State of
the Air

2

Technology Status

Currently in
Use

Prototype Exists

Under
Development

In Design

Concept State

3

Complexity

Simple

Somewhat
Complex

Moderately
Complex

Highly Complex

Highly Complex
with
Uncertainties

4

Interaction/
Dependencies

Independent
of Other Risk
Drivers

Dependent on
One Additional
Risk Driver

Dependent on
Two Additional
Risk Drivers

Dependent on
Three
Additional Risk
Drivers

Dependent on
more than Three
Additional Risk
Drivers

5

Producibility

Established

Demonstrated

Feasible

Known
Difficulties

Infeasible

6

Process Controls

Statistical
Process
Controls (SPC)

Documented
Controls
(No SPC)

Limited
Controls

Adequate
Controls

No Known
Controls

7

Manufacturing
Precision

High

Adequate

Limited
Margins

Known but
Inadequate

Unknown

1

Developed by F. D. Maxwell at the Aerospace Corporation. Included in “Estimating Cost Uncertainty when only Baseline Cost is
Available,” quoting R.L. Abramson and S. A. Book, “A Quantification Structure for Assessing Risk-Impact Drivers,” Laserlight
Networks, briefing presented to the 24th Annual DOD Cost Symposium (Leesburg, VA, September 5-7, 1990).

Section Revision:
May 2, 2016

5.2.7-7

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Risk Driver Category

Very Low

Low

Medium

High

Very High

8

Reliability

Historically
High

Average

Known Limited
Problems

Serious
Problems of
Unknown
Scope

Infeasible

9

Criticality to Mission

Nonessential

Minimum
Impact

Known
Alternatives
Available

Possible
Alternatives
Exist

“Show Stopper”

10

Cost

Established

Known History
or Close
Analogies

Predicted by
Calibrated
Model

Out of Range of
Experience

Unknown or
Unsupported
Estimate

11

Schedule

Demonstrated

Historical
Similarity

Validated
Analysis

Inadequate
Analysis

Unknown or
Unsupported
Estimate

5.2.7.6 Qualitative Risk Analysis – Risk Level Matrix
Once the probability and impact level of each identified risk is agreed to, the risk’s position is
determined on the probability and impact matrix shown in the following figures. The vertical
matrix axis labels in the figures correspond to the definitions for probability levels given in Table
5.2.7-1, and the horizontal axis labels correspond to the values for impact defined in Table
5.2.7-2. Combinations of probability and impact for a risk are shaded as red, yellow and green
for high, medium, and low risk level. The risk manager, project manager, or other stakeholder
should set these regions for each risk level, based on an understanding of the definitions of the
axes, which would cause the risk to rise to the appropriate level of attention.
Figure 5.2.7-3 shows a risk probability and impact matrix for one objective that is symmetrical.
Figure 5.2.7-4 shows a risk probability and impact matrix for an objective that emphasizes the
impact of the risk on its red-yellow-green status. This asymmetrical risk matrix indicates that
any risk that has a very high impact will achieve “high risk” or “red risk” status without regard to
the probability that the risk will occur on the project. Of course the definitions of risk impact
and probability buckets defined in the RMP will determine the relative score that the risk
achieves.

Section Revision:
May 2, 2016

5.2.7-8

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Figure 5.2.7-3

Symmetrical Risk Level Matrix

Probability and Impact Matrix for a Specific Objective (Time, Cost, Scope or Quality)
Probability
Symmetrical
Very High
Mod
Mod
High
High
High
High
Low
Mod
High
High
High
Mod
Low
Mod
Mod
High
High
Low
Low
Low
Mod
Mod
Mod
Very Low
Low
Low
Low
Low
Mod
Very Low
Low
Moderate
High
Very High
Impact

Figure 5.2.7-4

Asymmetrical Risk Level Matrix

Probability and Impact Matrix for an Objective (Time, Cost, Scope)
Probability
Impact Averse
Very High
Low
Mod
High
High
High
High
Low
Mod
Mod
High
High
Mod
Low
Low
Mod
High
High
Low
Low
Low
Mod
Mod
High
Very Low
Low
Low
Low
Mod
High
Very Low
Low
Moderate
High
Very High
Impact
The Risk Level for each objective for a defined risk depends upon where it falls in the Risk Level
Matrix according to the axes definitions. For example, a cost risk with an estimated probability
of 70% of occurrence and an estimated impact of $280K, or cost increase of 9% for the item at
risk, would fall into the High probability range and the Moderate cost impact range, according
to Table 5.2.7-1 and Table 5.2.7-2. Thus the Risk Level for cost for this particular risk occurring
falls into the High, or “red” range in Figure 5.2.7-3.
It is important to scale probability and impact so that the risks can be distinguished. On the one
hand, if the lower bound for an impact score of very high is easy to reach there will be many
risks with the same “red” assessment and the method will not distinguish risks for priority Risk
Handling. On the other hand, definitions of high or very high impacts that are very difficult to
reach will lead to very few or no “red” risks. While that may be true for some projects it would
be unusual for an NSF project with a high scientific impact.

Section Revision:
May 2, 2016

5.2.7-9

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

The objective of the matrix is to communicate the choice of priorities for monitoring and
response, which may best be done with the 2-D diagrams (5x5) shown herein. Depending upon
the activity and the ability to differentiate the risk levels, other matrices may be chosen by the
risk analysis team. For example, a 5x5x4 risk level (the fourth level represents the 4 objectives)
matrix would then have five values for probability, five for consequence and four for objectives.
Recall that Risk Management entails the identification and ranking of opportunities as well as
threats.1 Opportunities that are assessed to be in the “High” category are viewed as “lowhanging fruit” that can be easily claimed for the project if sought. For instance, if people are
coming off another similar project where they have had a good result, our project will benefit if
we can encourage or otherwise get them to join our project team. However, if such an
opportunity is not recognized in a timely manner, those productive people will go to other
projects. Another example is a potential cost saving if older but acceptable technologies can be
used in place of cutting edge solutions without impacting performance or quality. This type of
cost savings is common in data acquisition and storage systems, for instance.
The butterfly or mirror risk probability and impact matrix below shows scoring threats and
opportunities in similar ways. The red-yellow-green ranges for threats have been discussed. The
red risks for opportunities are those that have a high likelihood of occurring and if they occur
they help the project achieve its objectives, if only by offsetting threats. Risk Response of
opportunities needs to be proactive in order to secure these opportunities for the project.
Figure 5.2.7-5

Prob.
VH
H
M
L
VL

Probability and Impact Matrix including Threats and Opportunities

Mirror or Butterfly Probability and Impact Matrix for Threats and Opportunities
Project Objective (e.g., time, cost, scope, quality)
Threats
Opportunities
L
M
M
H
H
H
H
M
M
L
L
L
M
H
H
H
H
M
L
L
L
L
M
H
H
H
H
M
L
L
L
L
L
M
H
H
M
L
L
L
L
L
L
L
M
M
L
L
L
L
VL
L
M
H
VH
VH
H
M
L
VL
Impact (threats)
Impact (opportunities)

Prob.
V
H
M
L
VL

1

An early discussion of the use of opportunities in project management can be found in: Effective Opportunity Management for
Projects: Exploiting Positive Risk, David Hillson, Marcel Dekker, 2004.

Section Revision:
May 2, 2016

5.2.7-10

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.7.7 Risk Level Input to the Risk Register
The Risk Levels per each objective for all identified risks are entered into the Risk Register. See
the sample Risk Register in Figure 5.2.7-6. It is common practice to also include a column in the
matrix for the probability descriptor for ease of reference. As mentioned before, it is good
practice to list the Risk Level for each project objective separately and not combine them into a
single risk level for the stated risk. Projects may choose, however, to designate a flag to identify
some risks as “Major” or “Top” risks. These Top Risks are judged by the project management to
call for more aggressive management and more frequent monitoring than other risks.
Communicating and tracking the status of the top project risks is a key element of project
management. The Risk Management Plan should address the frequency with which these
significant risks are tracked. Top risks should be reviewed and evaluated during standard subsystem team meetings and reviews as well as at project status meetings.
Projects should also include a status report for the top risks in the various required reports to
NSF, including the monthly report, as well as for reviews. One simple method for
communicating the summary status of top risks to various stakeholders is shown in the sample
Top Risk Matrix shown in Figure 5.2.7-6, which shows risk level and trend data for selected
risks.

Section Revision:
May 2, 2016

5.2.7-11

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 5.2.7-6

Sample Risk Register with Risk ID Number, Associated WBS Identification, Qualitative Probability and Impact for Initial and Post-mitigation States, and
Mitigation Actions
Pre-Mitigated Scores
Probability and Impacts

Post-Mitigated Scores

Resulting Risk Score

Risk
ID

Risk
Description

Associated
WBS

Prob.
Risk
Occurs

On
Sched.

On
Cost

On
Scope/
Quality/
Performance

Risk
on
Sched.

Risk
on
Cost

Risk on
Scope/
Quality/
Performance

PM1

If …., then …

2.2

H

VH

H

M

H

H

TECH1

If …., then …

3.2.5

H

VH

H

H

H

EXT8

If …., then …

3.1.3

M

VH

H

M

H

ORG4

If …., then …

1.2

M

H

M

M

PM4

If …., then …

2.4.1

M

H

H

M

TECH5

If …., then …

3.3.1

Risk Action
Risk
Owner

Risk
Mitigation
Actions

Probability and Impacts
Trigger
or
Watch
date

Major
Risk
Flag

Resulting Risk Score

Prob.
Risk
Occurs

On
Schedule

On
Cost

On
Scope/
Quality/
Performance

Risk
on
Sched.

Risk
on
Cost

Risk on
Scope/
Quality/
Performance

M

M

M

M

L

M

M

M

H

H

M

H

M

L

H

M

M

H

M

L

H

M

L

M

M

L

H

M

M

L

M

M

M

M

M

M

H

H

M

L

L

L

L

L

L

L

M

VH

H

M

H

M

M

VL

M

M

M

L

M

L

TECH6

L

H

M

M

M

M

M

VL

L

L

M

L

L

M

EXT6

L

H

H

L

M

M

L

VL

M

L

L

M

L

L

PM2

M

L

H

L

M

H

M

L

L

H

VL

L

M

L

TECH9

VL

VH

VH

L

M

H

L

VL

H

VH

L

M

H

L

TECH10

VL

VH

M

VL

M

L

L

VL

M

L

VL

L

L

L

Section Revision:
May 2, 2016

5.2.7-12

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 5.2.7-7 Sample Top Risk Matrix and Status Report, showing list of project risks selected as most significant to monitor on a frequent basis, with ranking and trend data
Note that top risks include some with Low and Medium criticality (or ranking), as well as those evaluated as High criticality.

Project Top Risk Matrix
Trend Rank
5

L
LI
KI
K
E
E
L
HI
O
H
O
O
D
O
D

RSK-025
RSK-025
RSK-105

4

SOF-RSK-005

RSK-065
RSK-101
RSK-076
RSK-103

3

SOF-RSK-007

RSK-066

2

RSK-053

Risk ID

Approach

RSK-098

SOF-RSK-003,

RSK-102

SOF-RSK-010

RSK-106

SOF-RSK-009
RSK-100

SOF-RSK-002,

RSK-059

SOF-RSK-006

1

(4X4) RSK-025

M

p

1

(4x4) RSK-098

M

Operations Staff Fatigue



2

(3X4) RSK-102

M

Lack of Adequate
Planning for data analysis
pipeline



4

(3x3) RSK-103

M

Lack of Data Extraction
and Analysis Tools



5

(4x2) RSK-105

M

Scientist support staff
resources



6

(2x5) RSK-059

M

Loss of single-point-failure
component



7

(3x3) RSK-076

M

Maintaining Science
Instruments



8

(2X4) RSK-100

M

Instrument support
structural Integrity



9

(3x3) RSK-101

M

Undocumented Hardware
Requirements



3

(3X4) RSK-106

M

Science Instrument
Hardware Change
Control



10

(2x3) RSK-066

W

Operations Staff
Retention

11

(2x3) RSK-053

W

Changes in Safety and
Reliability Requirements

RSK-051

2

3

4

5

CONSEQUENCE
CONSEQUENCE

Criticality
High
Med
Low

L x C Trend

Approach



Decreasing (Improving)

M – Mitigate

p

Increasing (Worsening)

W – Watch



Unchanged

A – Accept



New since last period

R – Research

Title
Inadequate Observatory
Performance for initial
operations



SOF-RSK-001

1

1

Section Revision:
May 2, 2016

LxC



5.2.7-13

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.7 Qualitative Risk Analysis – Risk Register Ranking
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.7.8 Other Qualitative Risk Analysis Methods
Expected monetary value, simulation, Bayesian probability theory, reliability, and the use of
decision trees or its inverse, failure mode effects and criticality analysis (FMECA) are other risk
analysis methods that are used in project management. They are not described in detail here
but may be researched using the given references.
Expected monetary value is the product of the risk event probability multiplied by the value of
the gain or loss that will be incurred. Schedule impacts and intangibles (i.e., a loss may put the
organization out of business) must be considered when using this approach. This method for
scaling contingency amounts does not take advantage of information about the range of
possible impacts or probabilities. It can only provide a mean value of the contingency, not some
other target level of confidence. It is not good for time risks or cost risks that have time risk
components.1
Any schedule of a real project can easily be handled using Monte Carlo simulation techniques,2
discussed in the next section on Quantitative Risk Analysis. Simulation uses a model of a system
such as the project schedule to simulate a project using Monte Carlo analysis. Monte Carlo
“performs” the project many times so as to provide a statistical distribution of calculated
results under many different scenarios, since in each scenario different risks may occur with
different combinations of impact. The use of Monte Carlo analysis to estimate the risk schedule
or cost distribution by statistically combining risk costs is illustrated in the next section.
A decision tree is a diagram depicting key interactions between decisions and associated events
and uncertainties as understood by the decision-maker.3 A FMECA is a bottoms-up version of a
decision tree, building up from the elements to the decisions. Either approach helps the analyst
to divide a problem into a series of smaller, simpler, and more manageable events that more
accurately represent reality to simplify decision-making.
Bayesian probability theory treats probability as a degree of belief or uncertainty in a given
statement. More information may be found in Foundations of Risk Analysis. 4

1

Integrated cost-schedule risk analysis is presented in Section 5.2.8, Quantitative Risk Analysis.

2

For schedule impact the organization should not use the Program Evaluation and Review Technique (PERT or the Method of
Moments) to represent project risk in schedules. This method underestimates risk for the type of projects addressed herein.
Refer to “Project Schedule Risk Analysis: Monte Carlo Simulation or PERT?” David T. Hulett, PM Network published by the
Project Management Institute, February 2000, pp. 43 ff
3

See Recommended Practice 85R-14, Use of Decision Trees in Decision Making, Association for the Advancement of Cost
Engineering International (AACEI), 2014, David T. Hulett principal author. Also, “Use Decision Trees to Make Important Project
Decisions,” David T. Hulett, Cost Engineering (published by AACEI, July / August 2014.
4

Pages 62 and 64 of Foundations of Risk Analysis by Aven.

Section Revision:
May 2, 2016

5.2.7-14

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.8 Quantitative Risk Analysis – Estimating Contingency
5.2.8.1 The Purpose of Quantitative Risk Analysis
Quantitative risk analysis can analyze the impact of all of the risks and uncertainties on the
project objectives of overall time and cost. Hence quantitative risk analysis can derive results
that qualitative risk analysis cannot provide, i.e. the likely finish date and project cost when all
risks are considered within a model of the entire project.
Quantitative risk analysis allows the risk analyst to estimate:




How likely is the project to meet its schedule and cost goals?
How much schedule and cost contingency is needed to achieve the project’s desired
level of certainty?
Which risks are causing any potential overrun and are thus high priority for risk
mitigation?

Quantitative risk analysis allows the analyst to estimate the finish date and cost of the project
based on a probability distribution created by applying Monte Carlo simulation to a project plan
such as the schedule, cost estimate or cost-loaded schedule. The inputs are uncertainty and
discrete risk events, although there may also be probabilistic branches, weather / calendar
effects and even conditional branches. The process for performing Quantitative Risk Analysis is
shown in Figure 5.2.8-1 below. Outputs are the estimated total cost and finish date and
associated contingency amounts above the baseline input cost and finish date.
Figure 5.2.8-1

Quantitative Risk Analysis Process

• Risk and uncertainty
descriptions and
calibration
• Project artifacts such
as schedule,
estimate
• Client and team
interviews for risk
data
• Other data
gathering, lessons
learned

Inputs

Process
• Assess the schedule and cost against best
practices
• Conduct in-depth confidential risk interviews for
probability, impact and activities / costs affected
• Calibrate and assign inherent uncertainty and
estimating error, providing for more uncertainty
for work performed in the future
• Run Monte Carlo simulation on the risk-assigned
project schedule or estimate
• Prioritize the risks to the project
• Mitigate the high-priority risks and create a
post-mitigated result

• Likelihood of
finishing on time
and budget
• Needed cost and
time contingency
amounts
• Identification of
high priority risks
and needed risk
mitigations

Outputs

A quantitative risk analysis requires an accurate, up-to-date schedule as well as up-to-date risk
data to be useful. The schedule used for analysis is often not the detailed Integrated Master
Schedule (IMS), but is a summary schedule that can be resource loaded.

Section Revision:
May 2, 2016

5.2.8-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

While software tools have made it relatively simple to run a Quantitative Analysis, the
preparation work for a simulation run can take significant time and effort. Often, projects use
Qualitative Analysis for month-to-month risk management and use Quantitative Analysis for
establishing a new baseline or calculating an updated Estimate at Complete (EAC).
There are several commercial packages available that provide tools and programs for
performing Quantitative Analysis using cost estimates and/or resource loaded schedules.
While NSF strongly recommends probabilistic analysis methods for estimating total project risks
and contingency amounts, it does not endorse or recommend any particular program or
product.
A typical result of a quantitative schedule risk analysis using one such commercial tool, in this
case a schedule risk analysis histogram of possible end dates, is shown in Figure 5.2.8-2. The
estimated ranges of impact of risks and uncertainties on the duration of scheduled activities
were fed into a Monte Carlo simulation program that generated a distribution of possible end
dates based on a resource loaded schedule. For the histogram below, the horizontal axis shows
the possible end dates. The right vertical axis shows the end dates for the confidence level
curve.1 The dotted lines on the plot represent the end dates for which the confidence level for
completion by that date is 50% and 80% respectively. For this example, the PMB end date is
11/20/2015. If the project elects to use the 80% confidence level, then the chosen project finish
date is 7/28/2016, indicating that the project needs to mitigate or provide contingency for an
additional 8.3 months of project duration beyond the baseline date.

1

NSF sets a required range for the confidence level unless an exception is requested and approved by NSF. See Section 4.2.5 for
details.

Section Revision:
May 2, 2016

5.2.8-2

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 5.2.8-2

Typical Result of a Quantitative Schedule Risk Analysis

Another typical output, from quantitative analysis of a resource-loaded schedule, is a time-cost
scatter diagram. Figure 5.2.8-3 plots cost on the y-axis against end date on the x-axis. A line is
drawn through the slope of the distribution. The plot illustrates the important fact that time
and cost are related. In this case, longer schedule activities with labor-type resources generate
higher cost.

Section Revision:
May 2, 2016

5.2.8-3

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 5.2.8-3

Time-Cost Scatter Diagram: Each data point represents one realization of the simulation

5.2.8.2 Key Elements in Quantitative Risk Analysis
The platform for quantitative risk analysis is the project cost estimate or project schedule. Since
most cost estimates are developed in a spreadsheet, a risk analysis of the project’s cost
estimate alone is often conducted using a software package that simulates a spreadsheet
model.1 Schedule risk analyses simulate a project schedule, so software that is able to simulate
schedules developed in the organization’s preferred scheduling package must be used.2
Integrated cost-schedule risk analyses involve a good-quality PMB schedule (i.e., without cost
or schedule contingency) with loaded resources representing the cost estimate attached to the
activities they support.
1

Two commonly-used packages are @RISK from Palisade Corporation and Crystal Ball from Oracle. (NSF does not endorse or
recommend any particular package.)
2

There are several schedule simulation packages available. Two of the schedule simulation packages with the most capabilities
are Polaris from Booz Allen Hamilton and Primavera Risk Analysis from Oracle. Others include Acumen RISK from Deltek, Risky
Project from Intaver Institute, @RISK for Project from Palisades, JACS from Tecolote and Full Monte from Barbecana. (NSF does
not endorse or recommend any particular package.)

Section Revision:
May 2, 2016

5.2.8-4

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

The elements of risk that may affect the cost, duration, or both cost and duration of a project
include uncertainty, identified discrete risk events, and possible discontinuous events.


Uncertainty represents inherent variability in predicting the outcome of future events.
The uncertainty may be from people and organizations’ inability to do the same thing
the same way reliably or from the fact that future events cannot be predicted with
complete accuracy. Uncertainty has a probability of 100% (since it is always present) and
an estimated range for duration or cost. The range often has a positive tail (opportunity)
and a negative tail (threat) such as -5% and +10%. These ranges represent the
confidence in the estimates of activity duration or cost element actually occurring as
estimated. The uncertainty ranges are often specified as a 3-point estimate with low,
most likely and high values for a specified distribution shape, often a triangular
distribution. For every iteration in a simulation, the software pulls a random impact
multiplier for each duration and/or each cost item from within the chosen distributions.
That value, say 1.07, is then multiplied by the activity duration or element cost in the
model to get the value to be used for that iteration.
o There may be asymmetry in the range of uncertainty since it is often easier to
overrun than underrun an estimated value. Hence the optimistic tail of the
distribution may not have as much probability as the pessimistic tail has. Also,
the most likely value may not be the assigned value in the schedule or estimate.
Hence a fairly typical uncertainty range could be .95, 1.05, and 1.15 – the middle
value implies that the duration or cost is most likely 5% higher than in the
baseline model.
o The range of uncertainty can also be used to cover potential, but as yet
unidentified, discrete risks that may surface later in the project than at the time
of analysis. The inability to identify discrete risks is common for events that occur
significantly later in the project or for activities that cannot yet be well defined.
Most often these uncertain risks can be addressed by allocating a wider range of
uncertainty to these durations or costs than to those assigned to better
understood activities occurring in the early years of the project. In this way the
generally higher level of uncertainty for durations and costs in the later years of
the project can be included in the risk analysis leading to the size of the
contingency reserve.
o Some types of activities have more inherent uncertainty than others. It may be
more difficult to make estimates of duration and cost for testing than for design,
whereas fabrication may be somewhere in between. Therefore, some categories
of activities may have wider uncertainty ranges than others. These activity-type
specific uncertainty bands are sometimes termed reference ranges.



Discrete Risk Events include those already identified and quantified in the Risk Register
as well as any that may be discovered when interviewing for risk data to use in the

Section Revision:
May 2, 2016

5.2.8-5

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

quantitative risk analysis. Discrete risks are specified by their probability and range of
impact if they happen to activity durations or cost elements.
o The probability determines the fraction of the Monte Carlo iterations that they
appear in.
o The impact range is related to the duration of the individual activities or size of
cost line items that they are assigned to. Hence the concept of impact range for
quantitative analysis is not the same as that used for qualitative risk analysis,
which is impact on the final date or total cost for the entire project.
o A risk can affect many activities or cost elements. Activities or cost elements can
be affected by more than one, sometimes many, discrete risks.
o Discrete risks can be represented by adding a risk to a cost element or schedule
activity or by specifying a multiplicative factor to apply to the estimated cost (risk
register method) or activity duration (risk driver method).


Discontinuous Risk Events are discrete events that can have consequences beyond
adding duration to existing activities or cost to an existing budget element. Technically
challenging projects such as NSF facilities typically have numerous discontinuous risks.
Capturing a complete list of these risks is critical to effective RM and project success. For
example, failing a qualifying test (or other discontinuous event) may require adding new
activities and cost to the schedule in order to recover from the event. These activities
and cost elements are almost certain not to be in the baseline schedule or cost estimate
since those artifacts are usually based upon success of the baseline plan.

5.2.8.3 Platforms for a Project Quantitative Risk Analysis
A project schedule risk analysis starts with a good-quality Critical Path Method (CPM) schedule:


The schedule can be a summary or roll-up of the detailed schedule of the project and
should not have any padding or contingency for risk. Estimated project end date and
schedule contingency duration are outputs of the risk analysis. The detailed project
schedule is not always a good candidate for risk analysis input since it usually has several
thousand activities and may be difficult to debug. That is, the detailed project schedule,
perhaps a contractor’s schedule, may not conform to scheduling best practices. 1 Hence,
and in recognition that a schedule risk analysis is a strategic analysis of the project,
summary or “analytical” schedules may be used instead of the detailed schedule. This
analytical schedule needs to represent all the work of the project (including contractor
and other participants such as the customer) and be validated against best CPM best
practices. It is recommended that the summary or analytical schedule format adheres to
the project WBS to facilitate reporting of contingency usage.

1

One source of complete scheduling best practices is the Government Accountability Office (GAO) Schedule Assessment Guide,
2015.

Section Revision:
May 2, 2016

5.2.8-6

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Characteristics of a schedule used for Quantitative Risk Analysis are:
1)
2)
3)
4)

It represents all the work of the project,
All logic links are established,
All constraints are appropriate, and
It is resource loaded.

Since the schedule validation process can require significant effort by project leaders, some
references1 recommend creating a 300-1000 line summary schedule from the project IMS and
resource loading it with a minimal number of summary resources. (Some multi-billion dollar
projects have been known to use as few as eight summary resources.) That methodology is
followed in the exercise demonstrated in the following case study.
A cost risk analysis starts with a complete, PMB cost estimate:


The PMB cost estimate is complete for all in-scope work but does not include any builtin “padding” or contingency for risk. The estimated cost contingency amount is an
output of the risk analysis. The cost estimate is usually specified in spreadsheet format
and may be simple or detailed. It is recommended that the summary or analytical
schedule format adheres to the project WBS to facilitate reporting of contingency usage.

An integrated cost-schedule risk analysis starts with a resource loaded schedule for a PMB with
cost and schedule estimates:


A schedule, either analysis or detailed level, that is loaded with resources. For the
purpose of a risk analysis the resources do not have to be detailed at the same level as
the Cost Book, but they do have to distinguish between time-dependent (e.g., labor,
rented equipment) resources that will cost more if their activities are longer and timeindependent (e.g., materials, purchased equipment) resources that may have variable
cost but not because of uncertainty in duration. Again, it is recommended that the
summary or analytical schedule format adheres to the project WBS to facilitate
reporting of contingency usage.

All quantitative risk analyses require:


Good quality risk data collected in the Risk Register but usually enhanced using good
interview techniques. Note that SMEs are often more willing to talk freely about
extreme good and bad possible risk results in confidential interviews.

1

David T. Hulett, principal author, Recommended Practice 57R-09, Integrated Cost and Schedule Risk Analysis Using Monte
Carlo Simulation of a CPM Model, AACEI, 2011.

Section Revision:
May 2, 2016

5.2.8-7

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)




A professional schedule risk simulation package1 that can perform a Monte Carlo risk
analysis simulation on a risk-loaded schedule.
An organizational culture that is committed to conducting an unbiased and realistic risk
analysis and to use its output, such as total risk to objectives or prioritized risk events to
be mitigated in order to improve the prospects of the project.

5.2.8.4 Case Study: Quantitative Risk Analysis2 Exercise
These steps will be illustrated with a simple case study of an integrated cost-schedule risk
analysis of design and fabrication of a space vehicle, as shown in the resource-loaded Gantt
chart schedule shown in Figure 5.2.8-4.
Figure 5.2.8-4

Resource Loaded Schedule Used for a Simple Case Study of an Integrated Cost-Schedule Risk
Analysis for Design, Fabrication, Testing, and Delivery of a Space Vehicle

1

There are several different software packages that can do this analysis. The package used for these charts and tables is
Polaris® from Booz Allen Hamilton.
2 David T. Hulett, principal author, Recommended Practice 57R-09, Integrated Cost and Schedule Risk Analysis Using Monte
Carlo Simulation of a CPM Model, AACEI, 2011.

Section Revision:
May 2, 2016

5.2.8-8

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

This is a project starting June 1, 2008, with a ship to launch site end date of November 20, 2015.
The project cost is estimated at $651.6 million.1 Resources are shown on the bar chart and
include mostly labor, with some equipment in the First Stage and Upper Stage Fabrication
activities.
In this case study the resources, as shown in Table 5.2.8-1, are few and summary.
Table 5.2.8-1

Resources for Quantitative Risk Analysis Example

Resource Name

Type

Preliminary Designers

Work

Detail Engineers

Work

Fabrication

Material

Integrators

Work

Integration Testers

Work

Specification Writers

Work

Unit Testers

Work

Fabricators

Work

5.2.8.5 Schedule Risk Analysis ‒ Uncertainty
The schedule risk analysis starts with uncertainty reference ranges, estimated by the project
SMEs. Recall that the probability for uncertainty occurring is 100%, and thus occurs for all
simulation iterations for all assigned durations. The ranges shown in Table 5.2.8-2 are the SMEs’
estimates of uncertainty in the task durations. Note that three of these imply that the SME
interviewees assess the “Most Likely” value to be greater than the durations in the schedule.
This may be because they view the schedule as being built with optimistic durations or that
more has been learned about activity durations, leading to a higher estimate of the “most
likely” durations. Although not shown here, their evaluation could also have resulted in lower,
mostly durations. The use of Risk Drivers allows these distributions to have both threat and
opportunity tails.

1

This schedule has been developed in Microsoft Project. Another popular scheduling package is Primavera P6 from Oracle.
Most schedule simulation packages can import projects from these two scheduling packages.

Section Revision:
May 2, 2016

5.2.8-9

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 5.2.8-2

Schedule Duration Risk Reference Ranges

Activity Category

Low

Most
Likely

High

Designers

0.90

1.00

1.20

Fabricators

0.95

1.05

1.20

Integrators

0.95

1.05

1.20

Requirement Writers

0.90

1.00

1.15

Testers

0.85

1.10

1.25

Notice that these are fairly narrow ranges that represent inherent variability, for instance, but
do not represent the impact of discrete risks on the activity durations. These ranges are applied
to the activities in the named categories by a triangular distribution, in this case, from which
the computer pulls at random a multiplicative factor that is applied to the schedule duration.
The example exercise demonstrated here uses 5,000 iterations because the software is fast, but
3,000 iterations would generally be enough.
Uncertainty ranges should be wider the further out into the future the work is being planned
and estimated. This is because it is harder to estimate durations or costs several years into the
future, since the work has not been contracted yet and may not actually be detailed with any
specificity. Also, there will be risks in the future that cannot be identified today as discrete risks
but should be provided for with wider uncertainty ranges.
The analysis is performed using the reference ranges. If the analysis stopped at this point with
just uncertainties, the schedule results would look like the histogram shown in Figure 5.2.8-5
below. The 80th percentile has been chosen as the target level of confidence for this example.
The target confidence level for actual projects is chosen by the project or the customer.1 The
related cost risk histogram shown in Figure 5.2.8-6 represents the effect of duration uncertainty
alone on the costs for time-dependent resource.

1

To show these results one software package, Polaris, was chosen. However these results can be achieved using Primavera Risk
Analysis, JACS, Risky Project and others.

Section Revision:
May 2, 2016

5.2.8-10

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 5.2.8-5

End Date Results for Schedule Duration Uncertainties

Figure 5.2.8-6

Cost Result for Schedule Duration Uncertainties

Section Revision:
May 2, 2016

5.2.8-11

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

The results can also be shown in tabular form, with the 5% and 95% values included to
determine if the total range is believable. For uncertainties alone, the results in Table 5.2.8-3
are believable.
Table 5.2.8-3

Results with Schedule Uncertainties Assigned

Schedule

Baseline

5%

50%

80%

95%

Dates

20-Nov-15

8-Feb-16

19-May-16

14-Jul-16

7-Sep-16

2.6

6.0

7.8

9.6

Months from Base
Cost

Baseline

5%

50%

80%

95%

Dollars (millions)

651.6

660.6

684.4

697.1

709.5

1%

5%

7%

9%

% above Base

Because this is an integrated cost-schedule risk analysis there is interest in the relationship
between time and cost. This is shown in a finish date – total cost scatter diagram shown in
Figure 5.2.8-7. The scatter plot has a dot for each of all 5,000 iterations. The scatter slope
indicates the positive relationship between time on the horizontal axis and cost on the vertical
axis. The curved line toward the top right of the scatter represents those combinations of cost
and schedule results that exhibit a 70% probability of meeting both objectives, given the
uncertainties applied to the cost-loaded schedule. The target of 70% confidence level for
budgeting and scheduling was chosen in this case since it is often used by several government
funded agencies, such as NASA.1

1

See: “Understanding the Joint Confidence Level (JCL) at NASA,” NASA Office of Evaluation at 9/4/14.

Section Revision:
May 2, 2016

5.2.8-12

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 5.2.8-7

Total Cost and End Date Scatterplot for Schedule Uncertainties

Note that the schedule uncertainty data in this example are assumed to be not correlated. If
they were correlated (i.e. if one is high in its range then the others would also be high in their
ranges), the extremes in cost and time would be greater and the correlation between time and
cost would be tighter than shown above. The analyst should explore whether the uncertainty
distributions should exhibit correlation or not. If so, then the analyst will want to exploit the
capabilities of the chosen analysis package to handle correlations.
5.2.8.6 Schedule Risk Analysis ‒ Discrete Risks Added as Drivers
The second step is to identify, calibrate and assign discrete risks to the project schedule. The
risks used in this example are applied to the categories of activities, including design,
fabrication, integration, testing and requirements. For this exercise the risks are given generic
names, but in an actual analysis the risks would be taken from the Risk Register and augmented
by risks discussed in the confidential risk interviews. The generic risks for this exercise, with
their probabilities are shown in the top section in Figure 5.2.8-8. One risk, “Organizational risk
affecting all,” has been selected to show its assigned impact range next to the triangle symbol
on the right: Min 0.85, mode 1.05, Max 1.3. The Organizational risk has a probability of 70% and
is assigned to all tasks since its impact is felt on everything. Although the description has not
been filled in for this exercise, the organizational risk could stem from “lack of ready access to

Section Revision:
May 2, 2016

5.2.8-13

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

key decision makers that can increase durations” or to “organizational red tape that could slow
decision making,” for example.
Figure 5.2.8-8

Schedule Risk Drivers – Organizational Risk

When these risk drivers are assigned to multiple tasks or activities, those activities’ durations
become correlated since (1) if the risk occurs it occurs for all activities to which it is assigned,
and (2) the multiplicative factor chosen for that iteration is applied to all of those activities. If
only one risk is involved the activities become 100% correlated. If other risks are also assigned
the correlation between activity durations is reduced. In this way the risk driver method models
how correlation occurs so SMEs do not have to guess at the correlation matrix. With the
addition of discrete risks to the analysis, the schedule impacts are more pronounced and the
results show a later start (by 15.4 months) and higher cost (by $100 million) than with just the
uncertainties for the 80% confidence level. See Table 5.2.8-4 below. Note that the cost increase
is due to schedule duration risk drivers alone, and not to any cost uncertainty or risk.
The scatterplot in Figure 5.2.8-9 shows greater correlation of time and cost risk than the
previous plot showing uncertainties only, since the Organizational risk driver was assigned to all
activities.

Section Revision:
May 2, 2016

5.2.8-14

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 5.2.8-4

Results with Schedule Uncertainties and Discrete Risks Assigned

Schedule

Baseline

5%

50%

80%

95%

Dates

20-Nov-15

5-Jan-16

28-Dec-16

26-Oct-17

16-Aug-18

1.5

13.3

23.2

32.9

Months from Base
Cost

Baseline

5%

50%

80%

95%

Dollars (millions)

651.6

650.7

730.6

797.6

865

0%

12%

22%

33%

% above Base

Figure 5.2.8-9

Total Cost and End Date Scatterplot Showing Greater Correlation of Time and Cost Risk

5.2.8.7 Cost Risk Analysis ‒ Uncertainty and Discrete Risk Drivers
The last consideration in this simple example is whether there are uncertainties and discrete
risks for cost which would cause cost variations that are independent of schedule.
Examples of uncertainty could be errors in the time independent cost of fabrication, variances
in the time-dependent activities’ daily “burn rate” due to uncertainty in the number of
hours/workers needed per day, and/or uncertainty in the estimated salaries. These risks, if they
Section Revision:
May 2, 2016

5.2.8-15

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

occur, are in addition to the cost impact from schedule duration risks already discussed in the
previous material. The cost estimating error on the burn rate of labor or total cost of
equipment can be entered by resource as uncertainties, with probability of 100% and a range of
impact. Example uncertainty reference ranges for cost uncertainty as applied to different
resources for this exercise are shown in Figure 5.2.8-10.
Figure 5.2.8-10

Uncertainty in the Burn Rate and Total Cost
Resources and Their Utilization Uncertainty

UID

Planned Units per
Unit or Day

Rate Per Unit or Day
Min - Most Likely - Max

Resource

Type

1

Preliminary Engineers

Time Dependent

640

600 - 650 - 700

2

Detail Engineers

Time Dependent

960

900 - 960 - 1,020

3

Fabrication

Time Independent

1

0.9 - 1.05 - 1.15

4

Integrators

Time Dependent

1,200

1,100 - 1,250 - 1,500

5

Integration Testers

Time Dependent

1,200

1,150 - 1,250 - 1,550

6

Specification Writers

Time Dependent

800

750 - 800 - 850

7

Unit Testers

Time Dependent

800

700 - 825 - 950

8

Fabricators

Time Dependent

720

680 - 720 - 760

Discrete Risk drivers affecting cost can also be included to the analysis, in addition to the
uncertainty factors. These cost factors can be entered as the implication of identified risk
drivers, just as in the previous exercise for schedule drivers. If both cost and schedule risks
occur, the burn rate, cost estimate, and duration will vary, and each driver will cost to vary.
While new risks may be entered that just affect the burn rate or total cost of equipment, the
existing risks with schedule drivers already included can have those impacts as well. For
example, a cost factor has been added to the Risk Driver Editor for the previously identified
Organizational risk affecting all tasks, as shown in the Figure 5.2.8-11.

Section Revision:
May 2, 2016

5.2.8-16

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 5.2.8-11

Screenshot of Risk Driver Editor

After running the program with the addition of cost uncertainties to resources and allowing risk
drivers to affect costs directly rather than only through schedule risk, there is a direct impact on
cost, as can be seen in Table 5.2.8-5 below. The schedule table is not shown, since the cost
drivers included in the exercise do not by themselves impact duration. Note that some risks will
have just schedule duration uncertainties and risk drivers, some will have just cost uncertainties
and drivers, and some will have both. Cost will be affected in all cases, but schedule is affected
only for those risks with duration uncertainties and drivers.
Table 5.2.8-5

Results with Uncertainties and Cost Risks Assigned

Cost

Baseline

5%

50%

80%

95%

Dollars (millions)

651.6

679

838.6

975

1100

4%

29%

50%

69%

% above Base

Adding the uncertainty and risks affecting the costs independently of time to the simulations
results in a time-cost scatterplot shows less connection between time and cost, as shown in
Figure 5.2.8-12.

Section Revision:
May 2, 2016

5.2.8-17

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 5.2.8-12

Scatterplot Showing Less Connection between Time and Cost

5.2.8.8 Handling Inflation
Inflation is a part of the NSF budgeting and project planning. The program should select an
acceptable source for the future inflation rate and use it in the baseline and the risk analysis of
that baseline. For the case study in this exercise, the baseline cost is projected at $651.6 million
in base year dollars, that is, without inflation. With risks but no inflation the risk analysis
simulation shows a cost in base year dollars of $975 million at the 80th percentile of certainty.
The analysis program can be used to factor in inflation if the cost estimating has been
performed in base year dollars. Adding the factor of cost inflation and setting it at the rate of
3% causes the risked cost at the P-80 level to increase to $993 million in then-year dollars as
shown below in Figure 5.2.8-13:

Section Revision:
May 2, 2016

5.2.8-18

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Figure 5.2.8-13

Scatterplot with Addition of Cost Inflation Factor

The value of 3% inflation may be a most likely number, but the software used in this exercise
does not support an uncertain inflation level in simulation. A suggestion is to perform two
scenarios where the inflation rate is either lower or higher than 3%.



At 2% inflation the cost is estimated at $987 million
At 4% inflation the cost is estimated at $1 billion.

These scenarios can help understand the total “then dollar” cost of the project that is risk
adjusted, and the impact of the inflation assumption on that number.
5.2.8.9 Prioritizing the Discrete Risks ‒ Risk Mitigation Workshop
The organization is encouraged to use these results to help improve the prospects of the
project by mitigating the important risks. To do this the risks are prioritized. See Table 2.1.6-1
for sample prioritization from this exercise. This prioritization method uses the Monte Carlo
simulation, a 60-year old method, and the schedule which the project team is using to manage
or at least summarize the project. It is thought that this prioritization of risks is more realistic
than that using qualitative methods resulting in the risk register, in part because it recognizes
the structure of the schedule and handles correlations.

Section Revision:
May 2, 2016

5.2.8-19

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.8 Quantitative Risk Analysis – Estimating Contingency
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 5.2.8-6

Savings and Days Saved

ID

Name

Cost Savings

Days Saved

7

Organizational risk affecting all

$196.15M

207

7

Uncertainty

$48.03M

152

6

External risk affecting all

$34.97M

41

2

Risk affecting fabrication

$23.8M

31

4

Risk affecting testing

$4.28M

34

3

Risk affecting Integration

$10.86M

32

1

Risk affecting design

$5.35M

10

5

Risk affecting requirements

$0

0

The risk mitigation exercise must be done in a workshop setting since many people have to
contribute and commit to the mitigations.








This workshop includes the PM, DPM, team leads and others involved in mitigation of
risk.
Given the prioritized list of risks for a project that may overrun cost and schedule
targets, the project team can develop risk mitigation actions. The mitigation workshop
estimates the improvement in the probability and impact parameters is expected to
result from the various mitigations planned for each identified risk (uncertainty cannot
be mitigated in concept).
For the mitigation actions to “count” against the project risk management must commit
to them as evidenced by their post-mitigation budget, schedule and assignment of
people to monitor the risks and their mitigations. These risks should be added to the risk
register as well so they are reviewed frequently.
Each risk mitigation action accepted is modeled and the post-mitigation amount of risk
to time and cost is recorded, along with the cost of the risk mitigation. A post-mitigation
simulation will determine how much benefit is expected from the mitigations.
The final report includes post-risk mitigation results and the overall project cost and
schedule risk if those risk mitigation actions are taken and mitigate the risks. Note that
the original cost and schedule target will generally not be met since that would require
complete mitigation of the risks that caused the estimate of overrun in the risk analysis
itself.

Section Revision:
May 2, 2016

5.2.8-20

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.9 Risk Response Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.9 Risk Response Planning
Figure 5.2.9-1

Risk Response Planning Process

• Risk Register
• Meeting and
stakeholder
interviews with
dependent
organizations

Inputs

Process
• Confirm whether mitigation is required for risks based
on the Probability/Impact Ranking Chart
• Confirm root causes
• Develop a plan to mitigate the risk and provide a
Description and an Owner in the Risk Register
• Chart the risk mitigation events in time throughout
execution and map the improvement in risk status to
mitigation events
• WHEN: at the time risks are identified, at weekly/
monthly status meetings, and when changes to the
risks occur

• Updated Risk Register
• A separate action item
or schedule for
implementing risk
mitigation

Outputs

A known risk (often referred to as a “known unknown”) is a risk that has been identified and
can be calibrated (probability and impact) and analyzed. Examples of known risks may include
strategic or overriding aspects of the project environment such as poor project management
practices, lack of resources, multiple projects, external dependencies, relationships between
project participants, technical complexity etc. Identified risks need to be proactively managed
throughout the project life cycle by identifying who owns the management of that risk and by
outlining risk symptoms, triggers, and contingency plans that would prevent the risk from
occurring or that would lessen the project impact should it occur.
The Risk Response Planning step includes considerations related to risk mitigation and response
planning. This includes the assignment of one or more persons to take responsibility for each
identified risk and the development of measures and action plans to respond to the risk should
it become an issue. PMI PMBOK® Guide defines Risk Response Planning as the process of
developing options and actions to enhance opportunities and to reduce threats to project
objectives.
Risk response actions for threats are generally categorized as:1




1

Avoid – This strategy involves changing the project to eliminate the threat from
identified risk
Mitigate – This strategy involves taking early action to reduce the likelihood and/or
impact of risk
Transfer – This strategy involves shifting the responsibility and ownership of the risk to
another party. Although this strategy is seldom used for NSF projects, it typically

This listing and these descriptions are described in the PBMOK® Guide, 5th Edition, 2013 PMI

Section Revision:
May 2, 2016

5.2.9-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.9 Risk Response Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)



involves purchasing insurance against the type of risk or requiring vendors to assume
more risk.
Accept – This strategy involves acknowledging the threat as part of the project and
accepting the consequences of its occurrence. An example of this is political or
legislative risk that is out of the control of the project team. The consequence of
acceptance may mean that contingency resources may need to be applied if the risk is
realized.

Risk response actions for opportunities are generally categorized as:





Exploit – This strategy seeks to eliminate the uncertainty associated with this
opportunity to ensure it happens. This is similar to Avoid threats.
Enhance – This strategy seeks to increase the probability and / or the positive impacts of
the opportunity. This is similar to Mitigate threats.
Share – This strategy seeks to share the benefits of the opportunity with another
organization that is in the best position to secure the opportunity for the project. This is
similar to Transfer for threats.
Accept – This strategy accepts an opportunity if it arises but does not envision pursuing
it, similar to Accept for threats.

For the most part, project risk response planning will consist of defining risk thresholds for
action, confirming risk triggers, and then planning a mitigation strategy and/or developing
backup plans if risks occur. A risk trigger is an event or events that activate the execution of a
backup plan, should the risk become an issue. Triggers should be specified in the Risk Definition
in the Risk Register, as well as the date that risk resolution is required for each risk. Mitigation
strategies identify actions that may minimize or eliminate project risks before the risk occurs or
document decisions to accept the consequences of risks without action. A risk may have several
mitigation activities that attempt to balance the reduction in the probability and/or the severity
of the risk occurrence with the cost-effectiveness of the mitigation strategy. Mitigation planning
requires that the root cause(s) of the risk be identified and that the mitigation strategy and
plans be aligned accordingly. Backup plans define actions to be taken in response to identified
risk triggers in hopes of reducing potential project impact as a result of a realized risk (often
defined in the literature as an ”issue”).
A tabulated example of the impact of Risk Response evaluation is given below in Table 5.2.9-1.

Section Revision:
May 2, 2016

5.2.9-2

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.9 Risk Response Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)
Table 5.2.9-1

Impact of Risk Handling on Project Cost
Before Handling

Risk Item or Basis
Redesign to solve problems identified during reviews
Do analyses or design per external comments
Rework design documents during concept evolution
Redesign for add’l equipment for ops or pretreat interface
Design for sintering equipment
Redo design for SNF re-sizing
Redesign; contamination control in process room
Change design basis, due to scale-up impact
Redesign, for SC furnace
Redesign to add gas-trapping system
Rework to add waste streams to design
Rework robotic features design
Redesign for characterization
Redesign to meet canister requirements
Design for new cables
Redesign for additional MC&A equipment
Redesign, to apply new structural criteria to 105L
Redesign, per SGS inputs
Redesign for changes, per NRC interface
Additional utility design features
Delays initiating design, awaiting R&D completion
Delays redesigning for classified process control system
Add features to meet IAEA
Uncertainty in obtaining contingency funds
Disposal of bundling tubes
Decontamination of final-product canister
Storage location for depleted uranium
Availability of emergency generator and fuel tank
Redesign for necessary structural supports
Arithmetic Sums:
Section Revision:
May 2, 2016

Risk Level
Moderate
Moderate
Moderate
Moderate
High
Moderate
Moderate
Low
Low
Low
High
High
High
Moderate
Moderate
Moderate
Moderate
Low
Moderate
Moderate
High
Low
Moderate
Moderate
Moderate
Moderate
Moderate
Moderate
Moderate

Worst Case
Cost ($K)
3,360
390
5,720
160
500
200
5,000
50
800
1,550
3,000
7,440
5,000
3,000
400
400
1,500
500
200
500
5,360
60
500
2,000
100
500
100
40
300
48,630

After Handling
Residual Risk Cost Estimates ($K)
Handling
Strategy
Mitigate
Avoid
Mitigate
Mitigate
Mitigate
Accept
Mitigate
Accept
Mitigate
Accept
Mitigate
Mitigate
Mitigate
Accept
Mitigate
Mitigate
Mitigate
Accept
Mitigate
Accept
Mitigate
Avoid
Mitigate
Avoid
Avoid
Avoid
Avoid
Avoid
Avoid

Cost Implement
Handling
75
0
0
0
308
0
361
0
0
0
0
53
176
0
0
0
300
0
0
0
0
0
0
0
75
341
75
0
225
1,989

Risk
Level
Low
-Moderate
Low
Moderate
Moderate
Moderate
Low
Low
Low
Moderate
Moderate
Moderate
Moderate
Low
Low
Low
Low
Low
Moderate
Moderate
-Low
-------

Best
Case
0
N/A
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
N/A
0
N/A
N/A
N/A
N/A
N/A
N/A
0

Most
Likely
150
N/A
750
40
0
50
300
15
0
0
250
500
600
100
0
0
0
0
0
300
240
N/A
0
N/A
N/A
N/A
N/A
N/A
N/A
3,295

Worst
Case
500
N/A
2,500
100
200
200
3,000
50
50
1,550
2,300
2,000
3,000
3,000
50
50
700
500
150
500
720
N/A
50
N/A
N/A
N/A
N/A
N/A
N/A
21,170

5.2.9-3

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.9 Risk Response Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

The risks with mitigation plans and risk triggers are all listed in the Risk Register with their
Qualitative Risk Assessment status. After Risk Response Planning has been performed the entire
entry for the risk includes:








Its statement or definition
Analysis and ranking of initial risk
Assignment to a risk owner
Risk mitigation actions and backup plans
o Costs
o Timing and risk triggers
o Expected results
Status of mitigation efforts
Analysis and ranking of residual risk after mitigation

The last item listed above is the expected residual risk and ranking after mitigation has been
applied. This is accomplished by repeating the analysis of probability and ranked impact on
project objectives with successful mitigation assumed. Thus, the Risk Register shows “before”
and “after” views of the analysis, with risks migrating down from red to yellow to green with
the mitigation steps that cause the improvement in risk status and timing of those steps. The
sample risk register shown in Figure 5.2.7-6 shows columns with headings for “Pre-Mitigated”
and Post-Mitigated” analysis results.

Section Revision:
May 2, 2016

5.2.9-4

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.10 Risk Monitoring and Control
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.10 Risk Monitoring and Control
Risk Management requires continuous monitoring of project risk and iterative application of the
risk identification, analysis, and response processes. Existing risks need to be monitored,
controlled, and ultimately retired, while new risks must be identified and added to the Risk
Management process. Risk Monitoring and Control is the process of identifying, analyzing, and
planning for new risk, keeping track of and re-analyzing identified risks, monitoring risk
symptoms and triggers, reviewing the execution of risk responses strategies while evaluating
their effectiveness, and reporting status to stakeholders. The Risk Register, as the tool that
supports Risk Management and provides a means of communication, must be kept up to date
with status and changes. The frequency and process for reviewing project risk is set out in the
project Risk Management Plan.
Risk Control includes the process of regularly updating the Risk Register and communicating to
stakeholders the latest risk status, with resulting impacts on the project and mitigation plans.
Reporting of project and program level risks should be included as part of regularly scheduled
status meetings with, and in formal status reports to, internal project members as well as
eternal stakeholders and the NSF. The NSF has emphasized the need to communicate the risks
at regularly scheduled status meetings to ensure that continued focus and awareness is placed
on risk management.
Figure 5.2.10-1

Risk Monitoring and Control Process

• Risk Register
• Meeting and
stakeholder
interviews with
dependent
organizations
• Supporting
documentation
from project
artifacts

Inputs

Process
• Repeat risk identification to determine if new risks
should be put on the Risk Register and handled
• Regularly review the project for the risks reported on
the Risk Register to see if they have changed
probability or impact or should be retired
• Review the mitigation plans and validate that they still
represent an acceptable approach
• Disseminate mitigation strategies to stakeholders and
responsible resources. Follow on action items for
mitigation plans and report on status
• Use Risk Register to create and communicate the risk
disposition
• Ensure a continuous review of the risks and
monitoring for the mitigation and contingency efforts

• Updated Risk
Register
• Corrective actions
(not originally
identified as
mitigations)
• Report and
communicate on
new risks and
updated items

Outputs

When risks are resolved, they should be retired from the list of active risks. When the project
ends, the risk register may be closed. If some risks pose other future threats to the program or
future projects, consideration should be given to re-opening a risk with the appropriate
operations management, or at the program level as an “ongoing risk.”

Section Revision:
May 2, 2016

5.2.10-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.11 Contingency Management for Risk Mitigation
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.11 Contingency Management for Risk Mitigation
5.2.11.1 Contingency Budget Timeline
NSF expects the project to refine its WBS budget estimates following the Preliminary Design
Review (PDR), adding additional definition to the tasks associated with accomplishing the
project’s deliverable activities. At Final Design Review (FDR) the PEP budget estimate should be
substantially based on externally obtained cost estimates (vendor quotes, bids, historical data,
etc.). This added definition is expected to result in an increase to the project’s estimated
Budget at Completion (BAC) and project schedule, and a concomitant reduction in its budget
and schedule contingencies, while TPC and the risk-adjusted, committed schedule finish date
remain constant. The quantitative risk analysis should have a component to anticipate this
increase in cost and time so that the original contingency amounts are sufficient to provide for
this increase.
As a project progresses, the baseline cost estimate and schedule will typically be exceeded and
contingency amounts of dollars and time will be used. Periodically the project cost estimate
must be revised to reflect all new information, including actual costs and use of contingency
funds, adjustments to the risk profile, learning curves for manufactured items, etc. This new
estimate of the cost of the remaining work is called the Estimate to Complete (ETC), and the
Actual Cost of Work Performed + ETC is equal to the latest revision of the EAC. The EAC should
be compared to the sum of BAC plus remaining contingency to ensure that it is less than the
TPC. If the sum of BAC plus remaining contingency is greater than the TPC, de-scoping may be
necessary. See Section 4.2.5 for details on requirements for budget contingency use.
The project should create and maintain an expected contingency allocation profile. Contingency
allocation profiles usually do not track the commitment or spending profiles. For many projects,
the highest use of both schedule and budget contingency occurs during procurement and
during final commissioning/integration phases. A contingency allocation curve for such a
project would be bi-modal, with one peak for procurements activities and another for
significant contingency amounts held back until the end of the project, even though the
spending curve may be low near the end of the project. Although risk does burn down over
time, there may be significant reworking of hardware, for example, needed as a result of
knowledge gained during integration and commissioning activities.
5.2.11.2 Change Control for Contingency Adjustments
Adjustments to cost, schedule, and scope are documented and approved under the project
Change/Configuration Control Process (CCP), developed as a part of the PEP. The Risk
Management Plan describes how the project uses the Change/Configuration Control Process
(CCP) to assign contingency to specific WBS elements when risks materialize, and how budget
contingency is de-allocated from WBS elements and returned to the contingency category when
budget underruns occur. The Change Control Process should be initiated when the Total Project
Cost is established at the Preliminary Design Review, and followed for the duration of the
Section Revision:
May 2, 2016

5.2.11-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.11 Contingency Management for Risk Mitigation
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

project. All change control actions that affect the use of contingency – cost, schedule, or
technical performance and scope – must include a link to an identified and documented risk and
indicate the affected WBS elements at the first meaningful level of technical differentiation
within the project. The CCP must make provision for seeking prior approval from the NSF
Program Officer for all actions exceeding thresholds as defined in the CA. All change requests are
to be archived by the project, and made available for review by NSF. The Project must keep a log
of all change actions such that contingency actions, including puts and takes, can be reported
and summarized. See Section 4.2.5 for further details and a sample Change Request form.
Note that use of contingency does not automatically require a change to the baseline. For
instance, a change control action can authorize contingency to cover a cost overrun which is
tracked as a variance on the baseline Budget at Complete (BAC). In such a case the contingency
can be incorporated into either the BAC or the EAC. In the first instance, the BAC is changed. In
the second, the variance from the BAC remains and can be used for trending and other
information. See Section 4.2.5 for further details on approval levels for use of contingency.
Adjustments to contingency should include taking advantage of opportunities to assign savings
and underruns to contingency. Savings should not be left in associated WBS elements if they
are above thresholds set out in the Risk Management Plan, nor should they be shifted to other
tasks without going through the Change Control Process for return to contingency and
subsequent allocation to a different WBS element. Budget and cost underruns should be moved
to contingency as risks are retired and WBS elements are closed out and reconciled. Savings
realized through the implementation of planned de-scoping options should also be placed into
contingency. Returning the savings allows the best use of contingency for overall project
priorities.
5.2.11.3 Liens List: Forecasting and Opportunity Management
The Project should maintain a Liens List of planned future adjustments to contingency as a
forecasting tool that tracks actions that have not yet been incorporated into the BAC or EAC.
The list may document items such as very high probability risks with trigger points for action,
deferred scope held as contingency until a decision date, realized risks needing draws on
contingency that require more definition for a change control action to be implemented, and
anticipated opportunities for returns to contingency. It can also be used to record the need for
contingency to cover variances that will not/cannot be mitigated. It does not serve the same
purpose as a watch list or major threats list from the Risk Register. It acts as an escrow or
staging account for planned or near certain contingency allocations.
The List should include a description of the identified risk and the anticipated action, with
estimates of budget and schedule impacts, and anticipated decision date for any CCB action.
The affected WBS elements should be identified, at the second level (or the first meaningfully
specific level of scope description), where known.

Section Revision:
May 2, 2016

5.2.11-2

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.11 Contingency Management for Risk Mitigation
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Projected amounts of possible future adjustments to contingency in the Liens List are to be
periodically reported to NSF. NSF recommends including this information within the monthly
status report as well.
5.2.11.4 Updates of the Estimate at Complete and Risk Exposure
The project should maintain an estimate of total costs and risk exposure by periodically
updating the schedule, the Estimate at Complete (EAC), and the analysis of overall project risk.
Estimated contingency amounts should be appropriate for the risk exposure throughout the
project life cycle. During concept and early development stages, a qualitative risk analysis and
risk register may provide an adequate estimate of risk exposure for both the design and
construction planning estimates. As project planning reaches the preliminary design phase, the
drawbacks of qualitative analysis – limited subset of risks, ignored correlations, and arithmetic
sums of averages – do not allow that method to adequately portray total project risk.1 Project
planners must transition to quantitative risk analysis in order to establish a substantiated Total
Project Cost at the time of the PDR.
For the construction stage, initial contingency is a part of project total cost, scope and schedule.
As time goes by, risk exposure changes with risk mitigation, new knowledge, and new
circumstances. The amount of remaining contingency budget fluctuates over time with
assignments to risk mitigation and return of savings. The remaining risk exposure estimate
should be compared to the remaining available contingency to determine whether the project
has adequate funds to cover anticipated risks. Remaining available contingency (RAC) is defined
as the difference between the combination of the EAC plus any liens and The Total Project Cost:
RAC = Total Project Cost (TPC) - (EAC + liens)
The sum of the (EAC + liens) should include variances (backward looking actuals) and updated
estimates (forward looking forecasting) in the current plan, not the target baseline BAC. The
EAC should equal the BAC only at project start and after major changes to the baseline from replanning or re-baselining.
It is good practice to re-estimate EAC and Risk Exposure yearly, unless stated otherwise in the
CA. Specific dates may also be appropriate times for re-evaluation, such as at major milestones
dates. The Project Manager periodically re-assesses the current risk assessment to identify and
address any new risks that arise as the project progresses. This assessment should result in a
determination of whether cost and schedule contingency remains sufficient for project risks.

1

Projects usually adopt a more conservative certainty target such as the 80th percentile.

Section Revision:
May 2, 2016

5.2.11-3

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.11 Contingency Management for Risk Mitigation
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.11.5 Contingency Use and NSF Oversight during Construction
The NSF Program Officer must concur on all CCB actions exceeding CA-defined thresholds for
allocation of budget, schedule, or scope contingency. NSF will negotiate a CA with the recipient
institution to fund project construction activity which will specify thresholds above which prior
NSF approval1 is required before allocation of contingency (following formal CCB review) to
specific WBS elements. Contingency may only be used to support in-scope work for the
approved project baseline. See Section 4.2.5.7 for additional details.
5.2.11.6 Documentation and Reporting of Contingency Use
Risk management actions involving Change Control actions fall under the following
documentation and reporting requirements, as stated in more detail in Section 4.2.5:





All Change Control Requests, irrespective of amount or whether they increase or
decrease the BAC, are to be reported directly to NSF Program Officer
The recipient will keep an archive of all Change Control Requests
The recipient will keep a summary log of all Change Control Requests
Projected amounts of possible future adjustments to contingency (“liens”) are to be
periodically reported to NSF.

NSF recommends including this information within the monthly status report. Note that
National Science Board (NSB) approvals2 are required when Change Control actions exceed the
even higher thresholds defined by NSB policy.
The required summary log of all Change Control actions should include the following:









Change control action title,
Change control document reference number,
Change control approval date,
Amounts of change in budget, scope, and/or schedule, for each affected and identified
WBS element,
Any adjustments to contingency amounts,
WBS elements affected by the changes (at WBS Level II or at the first meaningful level
of technical differentiation within the project)
Risk Register ID number and description for the risk being addressed, and
NSF approval date if required.

1

Thresholds are necessary to allow the project to respond in a timely way to small, immediate needs for use of contingency,
such as field changes during construction. This avoids potential cost escalation that could result from delay.
2

See Section 2.4, Construction Stage, for details on NSF policy on how and when NSB approval is required.

Section Revision:
May 2, 2016

5.2.11-4

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.11 Contingency Management for Risk Mitigation
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Monthly reports must also include the status of contingency as part of the Earned Value
Management (EVM) reports. See Section 4.2.5.8 for details on reporting.

Section Revision:
May 2, 2016

5.2.11-2

Large Facilities Manual: NSF 16-0XX (May 2016)
5.2.12 Partnership Considerations for Contingency Management
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.2.12 Partnership Considerations for Contingency Management
NSF may partner with other entities to plan and construct a major facility. The guidelines within
Section 4.2.5.9 of this document are applicable when NSF funds a particular scope of work
within a larger overall project. Risk assessment, contingency development processes and
contingency status reporting are to be applied to those WBS elements to be funded by NSF.
NSF encourages the development of unified management for project planning and execution of
the entire project scope wherever practical.

Section Revision:
May 2, 2016

5.2.12-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.3 Guidelines for Cyber-Security of NSF’s Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.3

GUIDELINES FOR CYBER-SECURITY OF NSF’S LARGE FACILITIES

NSF has responsibility for oversight of facilities it constructs and operates, including associated
IT Infrastructure. This section, to be written, will describe what NSF considers to be a
fundamental set of IT security requirements that facilities should consider in developing and
deploying their IT plans, policies and procedures. These minimal requirements and their
associated evaluation criteria, as provided by the facility and agreed to by NSF, are used as part
of NSF’s facility oversight and review process. This module will document NSF’s expectation for
the recipient and PO oversight for the implementation and monitoring of cyber-security best
practices. These expectations extend over the full life cycle of an award, and are appropriately
modified as the award passes through various stages of its life cycle.

Section Revision:
June 17, 2015

5.3-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.4 Guidelines for Planning and Executing External Reviews of NSF's Large Facilities
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.4

GUIDELINES FOR PLANNING AND EXECUTING EXTERNAL REVIEWS OF NSF'S LARGE
FACILITIES

This document, which is in preparation, will describe the process for evaluation and review of
all NSF large facility projects proposed for construction, under construction or currently in
operation. It will provide assistance to the Program Officer (PO) in preparing and planning a
review of the non-research related aspect of the project’s management, budgets, schedule and
related activities. The information contained will offer guidance for three situations: reviews of
facilities in planning; reviews of construction activity; and operational reviews of ongoing
facilities. A description of the overall process of planning and carrying out an external review of
a large facility project is provided as an aid to the PO or associated staff who may be unfamiliar
with these processes or need a reference source on best practices.
The evaluation and reviews covered in the document include assessment of management,
schedules and budgets, as well as other matters relevant to a large facility project, such as
scrutiny of the project baseline for construction activity. It does not address the intellectual
merit or the broader impact criteria used to select the project for support, but rather focuses
on evaluation of the Recipient’s planning and implementation activities.

Section Revision:
June 17, 2015

5.4-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.5 Environmental Considerations in Large Facility Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.5

ENVIRONMENTAL CONSIDERATIONS IN LARGE FACILITY PLANNING

NSF’s funding for the construction or modification of facilities constitutes a Federal Action that
triggers compliance with several statutes designed to protect the Nation’s environmental,
cultural and historic resources. Awareness of, and strict adherence to, all relevant
environmental regulations are extremely important considerations in the planning,
construction and operation of facilities.
These statutes include, but are not limited to, the National Environmental Policy Act (NEPA),
the National Historic Preservation Act (NHPA) and the Endangered Species Act. Furthermore,
there are international agreements and treaties that deal with environmental impacts. For
further information, see the Proposal and Award Policies Procedures Guide Chapter II.C.2.j.
NSF regulations governing compliance with the National Environmental Policy Act (NEPA) are
found at 45 CFR §640. NSF regulations supplement the Council on Environmental Quality’s
(CEQ) regulations, published at 40 CFR §§1500-1508. Program Officers, as required by NSF
regulations, are responsible for evaluating the environmental impacts that may result from the
implementation of a Foundation award and determining into which category incoming
proposals fall (i.e., CATEX, EA, or EIS). Compliance with NEPA also includes providing
opportunities for public input on issues such as potential environmental impacts and ways to
avoid, minimize, and/or mitigate adverse impacts. Determining the required level of
compliance activities – including what documentation, consultation and/or permits may be
required – is a complex task. The Program Officer (PO) should not attempt to determine the
extent of compliance requirements without consulting the Environmental Compliance Team
within NSF's Office of the General Counsel. NEPA compliance may require the preparation of an
Environmental Assessment (EA) in cases where no significant environmental impacts are
expected or the more extensive documentation of an Environmental Impact Statement (EIS)
where adverse effects are anticipated. Failure to take necessary steps can cause undue delays
in a project’s schedule, significant cost escalation and potential federal litigation.
Additionally, in conjunction with or independent of its NEPA compliance, NSF may be required
to initiate consultations with Native Americans and other interested parties pursuant to the
NHPA and/or initiate informal or formal consultation with the U.S. Fish and Wildlife Service
under the Endangered Species Act. These compliance requirements can introduce significant
schedule and cost risk into the project which should be considered and addressed.
Furthermore, there is no special source of funding within NSF to pay for the environmental
compliance process; the cost is normally borne by the program using Research and Related
Activities (R&RA) funds.
Given the above considerations, the following guidance is offered:

Section Revision:
May 2, 2016

5.5-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.5 Environmental Considerations in Large Facility Planning
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

1. It is imperative that the PO contact the Environmental Compliance Team within NSF's
Office of the General Counsel early in the conceptual design stage to seek guidance on
specific requirements for compliance.
2. It is extremely important that the PO and the project get cost estimates for the
compliance process and factor these into the project’s scope, schedule and budget early
in the design process.
The cost drivers associated with these activities (their impact on the project construction cost)
need to be well understood by PDR since the PDR budget and risk assessment provide the basis
for the construction funding request.

Section Revision:
May 2, 2016

5.5-2

Large Facilities Manual: NSF 16-0XX (May 2016)
5.6 Guidelines for Property Mangement
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.6

GUIDELINES FOR PROPERTY MANGEMENT

(Reserved for future content)

Section Revision:
May 2, 2016

5.6-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.7 Guidelines for Financial Management
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.7

GUIDELINES FOR FINANCIAL MANAGEMENT

(Reserved for future content)

Section Revision:
May 2, 2016

5.7-1

Large Facilities Manual: NSF 16-0XX (May 2016)
5.8 Guidelines for Earned Value Mangement Systems
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

5.8

GUIDELINES FOR EARNED VALUE MANGEMENT SYSTEMS

(Reserved for future content)

Section Revision:
May 2, 2016

5.8-1

Large Facilities Manual: NSF 16-0XX (May 2016)
6 References
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

6

REFERENCES

NSF Reference Documents
Award Monitoring and Business Assistance Program [AMBAP] … guide
Director’s Review Board Procedures (O/D 97-07; June 10, 1997): O/D 97-07
Empowering the Nation Through Discovery and Innovation: NSF Strategic Plan for Fiscal Years
(FY) 2011-2016;
“Guidelines for Development of Internal Management Plans for Large Facilities” (an internal
NSF document)
Moses, Lincoln E., Administrator of the Energy Information Administration, Administrator’s
Message to the Annual Report to Congress, 1977, Volume Three
NSF Business Systems Review (BSR) Guide
NSF Facility Plan
NSF Governance and Management for the Future, a report by a panel of the National Academy
of Public Administration, April 2004.
NSF Proposal and Award Manual
NSF Proposal and Award Policies and Procedures Guide
NSF’s 2009 Budget Request to Congress, page 3, available online
Suite of NSF Terms and Conditions: The NSF website on How to Manage Your Award
http://www.nsf.gov/awards/managing/

NSB Reference Documents
Joint National Science Board —National Science Foundation Management Report: Setting
Priorities for Large Facility Projects Supported by the National Science Foundation (NSB05-77); September 2005
NSB MREFC Process (graphic, NSB-/CPP – 12-18, approved May 4, 2012)
NSB Statement on Competition, Recompetition, and Renewal of NSF Awards, NSB 08-16, NSB
Statement on Competition, Recompetition, and Renewal of NSF Awards, NSB 08-16,
https://www.nsf.gov/nsb/publications/2008/nsb0816_statement.pdf

Section Revision:
May 2, 2016

6-1

Large Facilities Manual: NSF 16-0XX (May 2016)
6 References
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

NSB’s Annual Timeline for Integration of Board MREFC Process with NSF Budget Process (NSB10-66, approved August 2010)
Priority Setting for Large Facility Projects (NSB-04-96), National Science Board White Paper,
May 2004, Attachment 5 to NSB Meeting Report,
http://www.nsf.gov/nsb/meetings/2004/may_srprt.doc.
418th Meeting of the National Science Board, Open Session Approved Minutes, Resolution NSB11-1, Adopted February 16, 2011
Science and Engineering Infrastructure Report for the 21st Century -The Role of the National
Science Foundation, February 8, 2003. NSB-02-190,
http://www.nsf.gov/nsb/documents/2002/nsb02190/nsb02190.pdf
Setting Priorities for Large Research Facility Projects Supported by the National Science
Foundation, a 2004 National Academies report
US Government Acts and Laws References
42 U.S. Code 1873(b), which states “The Foundation shall not, itself, operate any laboratories or
pilot plants.”
Endangered Species Act
Government Performance and Results (GPRA) Modernization Act of 2010 (P.L. 111-352)
National Environmental Policy Act (NEPA)
National Historic Preservation Act (NHPA)
NSF’s Authorization Act of 2002, 42 U.S.C.1862n-4(c), signed into law on December 19, 2002,
Public Law 107-368, Section 14(c)
General Facility Management Guides
“Best Practices for Federal Research and Development Facility Partnerships,” IDA Science &
Technology Policy Institute, IDA Paper P-5148 Log: H 14-000676
Guide to the Project Management Body of Knowledge, (PMBOK® Guide), Project Management
Institute, 5th Edition, 2013
Cost and Schedule Estimating and Risk Analysis References
Aven, Foundations of Risk Analysis

Section Revision:
May 2, 2016

6-2

Large Facilities Manual: NSF 16-0XX (May 2016)
6 References
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Butts, Glenn and Kent Linton, “NASA the Joint Confidence Level Paradox – a History of Denial,”
NASA Cost Symposium, April 28, 2009
“Estimating Cost Uncertainty when only Baseline Cost is Available,” quoting R.L. Abramson and
S. A. Book, “A Quantification Structure for Assessing Risk-Impact Drivers,” Laserlight
Networks, briefing presented to the 24th Annual DOD Cost Symposium (Leesburg, VA,
September 5-7, 1990)
Federal Acquisition Regulations (FAR Part 35.017-4, Reviewing FFRDCs)
Flyvbjerg, Bent, Nils Bruzelius, and Werner Rothengatter, Megaprojects and Risk: An Anatomy
of Ambition, 2003 Cambridge University Press
Government Accountability Office (GAO) Cost Estimating and Assessment Guide, (GAO-09-3SP
2009)
Government Accountability Office (GAO) Schedule Assessment Guide (GAO-16-89G 2015)
Hillson, David, Effective Opportunity Management for Projects: Exploiting Positive Risk, Marcel
Dekker, 2004
Hillson, David and Ruth Murray-Webster, Understanding and Managing Risk Attitude, Gower,
2005
Hillson, David Private research conducted in 2004 and presented at a PMI EMEA conference
Hulett, David T., “Project Schedule Risk Analysis: Monte Carlo Simulation or PERT?” PM
Network published by the Project Management Institute, February 2000, pp. 43 ff
Hulett, David T., “Use Decision Trees to Make Important Project Decisions", Cost Engineering
(published by AACEI, July / August 2014)
Hulett, David T., Integrated Cost-Schedule Risk Analysis, Cost Engineering (AACE International),
November/December 2012, pp. 5-15, with Michael Nosbisch
Hulett, David T., principal author, Recommended Practice 57R-09, Integrated Cost and Schedule
Risk Analysis using Monte Carlo Simulation of a CPM Model, AACEI, 2011
Hulett, David T., principal author, Recommended Practice 85R-14, "Use of Decision Trees in
Decision Making", AACEI, 2014
Hulett, David T., “Use Decision Trees to Make Important Project Decisions,” Cost Engineering
(published by AACEI, July / August 2014
Hulett, David T., Integrated Cost-Schedule Risk Analysis, Gower Publishers, 2011

Section Revision:
May 2, 2016

6-3

Large Facilities Manual: NSF 16-0XX (May 2016)
6 References
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Hulett, David T., Practical Schedule Risk Analysis, Gower Publishers, 2009
Merrow, Edward W., Industrial Megaprojects, 2011, Wiley
NASA's Challenges to Meeting cost, Schedule, and Performance Goals, NASA IG-12-21
Pariseau and Oswalt, Using Data Types and Scales for Analysis and Decision Making
Planning, Budgeting, Acquisition, and Management of Capital Assets, OMB Circular No. A–11
(2014).
“Understanding the Joint Confidence Level (JCL) at NASA,” NASA Office of Evaluation at 9/4/14.

Section Revision:
May 2, 2016

6-4

Large Facilities Manual: NSF 16-0XX (May 2016)
7 List of Acronyms
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

7

LIST OF ACRONYMS

AD
AMBAP
BAC
BFA
BOE
BOT
CA
CAAR
CATEX
CCB
CCP
CDR
CEP
CO
CPM
CPRD
CSB
DACS
DD
DDLFP
DGA
DIAS
DRB
EA
EAC
EHR
EIS
ES&H
ETC
EVM
EVMS
FAR
FDR
FFRDC
FY
GAO
G/AO
GPRA
HLFO
ICA
ICE

Section Revision:
May 2, 2016

Assistant Director
Award Monitoring and Business Assistance Program
Budget at Complete
Office of Budget, Finance, and Award Management
Basis of Estimate
Business Oversight Team (Replaced by Integrated Project Team [IPT])
Cooperative Agreement
Cost Analysis and Audit Resolution
Categorical Exclusion (NEPA)
Change Control Board
Change/Configuration Control Process
Conceptual Design Review
Cost Estimating Plan
Contracts Officer
Critical Path Method
Cost Proposal Review Document
Cooperative Support Branch
Division of Acquisition and Cooperative Support
Division Director
Deputy Director for Large Facility Projects
Division of Grants and Agreements
Division of Institution and Award Support
Director’s Review Board
Environmental Assessment
Estimate at Complete
Education and Human Resources
Environmental Impact Statement
environmental safety and health
Estimate to Complete (for Cost)
Earned Value Management
Earned Value Management System
Federal Acquisition Regulations
Final Design Review
Federally Funded Research and Development Center
Fiscal Years
Government Accountability Office
Grants and Agreements Officer
Government Performance and Results Act
Head, Large Facilities Office
Independent Cost Assessment
Independent Cost Estimate

7-1

Large Facilities Manual: NSF 16-0XX (May 2016)
7 List of Acronyms
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

IMP
IMS
IPT
IT
LFM
LFO
LFWG
MOU
MREFC
NCE
NEPA
NHPA
NICRA
NRC
NSB
NSF
O&M
OD
OGC
OMB
ORR
PAM
PAT
PDP
PDR
PEP
PI
PMB
PMBOK
PO
R&D
R&RA
RAC
RBS
RET
REU
RFP
RI
RM
RMP
S&E
SOG
SME
Section Revision:
May 2, 2016

Internal Management Plan
Integrated Master Schedule
Integrated Project Team (replaces the BOT and the PAT)
Information Technology
Large Facilities Manual
Large Facilities Office at NSF
Large Facilities Working Group
Memorandum of Understanding
Major Research Equipment and Facilities Construction
No-Cost Extension
National Environmental Policy Act
National Historic Preservation Act
Negotiated Indirect Cost Rate Agreement
National Research Council
National Science Board
National Science Foundation
Operations and Maintenance
Office of the Director
Office of the General Counsel
Office of Management and Budget
Operations Readiness Review
Proposal and Award Manual
Project Advisory Team (replaced by the Integrated Project Team [IPT])
Project Development Plan
Preliminary Design Review
Project Execution Plan
Principal Investigator
Performance Measurement Baseline
Project Management Body of Knowledge
Program Officer
Research and Development
Research and Related Activities
Remaining Available Contingency
Risk Breakdown Structure
Research Experiences for Teachers
Research Experiences for Undergraduates
Request for Proposals
Research Infrastructure
Risk Manager
Risk Management Plan
Science and Engineering
Standard Operating Guidance
Subject Matter Expert
7-2

Large Facilities Manual: NSF 16-0XX (May 2016)
7 List of Acronyms
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

SPC
TPC
WBS

Section Revision:
May 2, 2016

Statistical Process Control
Total Project Cost
Work Breakdown Structure

7-3

Large Facilities Manual: NSF 16-0XX (May 2016)
8.1 Lexicon Preface
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

8
8.1

LEXICON
LEXICON PREFACE

This Lexicon contains definitions of project and program management terms used in this
Manual, as applied to NSF large facilities. It is a combination of specialized terms defined by NSF
and used in the management of its large facilities, and terms and definitions commonly used in
professional project and program management. A subset of common project management
terms compatible with NSF usage were selected from a standard source, the PMI Lexicon,1 for
inclusion in this lexicon.
The Lexicon provides a common set of standard terms and definitions that should facilitate
communication and understanding between stakeholders when used in documents and
correspondence related to large facility management.
The terms and definitions included in this lexicon are in development and are subject to
modifications in future versions.

Entries in italics in this lexicon have been reproduced with permission from Project Management Institute, Inc.,
[PMI Lexicon], (2012) Copyright and all rights reserved.
1

Section Revision:
June 17, 2015

8-1

Large Facilities Manual: NSF 16-0XX (May 2016)
8.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

8.2

TERMS AND DEFINITIONS

Acceptance Criteria. A set of conditions that is required to be met before deliverables are accepted.
Activity. A distinct, scheduled portion of work performed during the course of a project.
Actual Cost. The realized cost incurred for the work performed on an activity during a specific time
period.
Analogous Estimating. A technique for estimating the duration or cost of an activity or a project, using
historical data from a similar activity or project.
Apportioned Effort. An activity where effort is allotted proportionately across certain discrete efforts
and not divisible into discrete efforts. (Note: Apportioned effort is one of three earned value management
[EVM] types of activities used to measure work performance.)

Approval. The act of officially accepting an idea, action, or plan.
Assistance. The act of giving support or help; making it easier for someone to do something or
for something to happen.
Assumption. A factor in the planning process that is considered to be true, real, or certain, without
proof or demonstration.

Assurance. To give a strong and/or definite statement that something will happen or that
something is true; to give confidence to.
Backward Pass. A critical path method technique for calculating the late start and late finish dates by
working backward through the schedule model from the project end date.
Baseline. The approved version of a work product that can be changed only through formal change
control procedures and is used as a basis for comparison.

Baseline Definition. The description of the approved scope of work and resources for a
construction project, including a hierarchical, product-oriented Work Breakdown Structure
(WBS) and associated WBS dictionary; the cost and schedule Performance Measurement
Baselines (PMB); and any contingency amounts.
Basis of Estimate. Supporting documentation outlining the details used in establishing project estimates
such as assumptions, constraints, level of detail, ranges, and confidence levels.

Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.

Section Revision:
May 2, 2016

8-2

Large Facilities Manual: NSF 16-0XX (May 2016)
8.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Budget Contingency. An amount added to a baseline budget estimate to allow for identified
items, conditions, or events for which the state, occurrence, or effect is uncertain and that
experience shows will likely result, in aggregate, in additional costs. Typically estimated using
statistical analysis or judgment based on past asset or project experience. Budget contingency
is allowed on MREFC awards.
Bottom-up Estimating. A method of estimating project duration or cost by aggregating the estimates
of the lower-level components of the work breakdown structure (WBS).
*Budget at Completion. The sum of all budgets established for the work to be performed. (For NSF

projects, contingency amounts are not included in the ETC, EAC, BAC, or PMB due to the NSF
requirement that contingency is held and managed separately from the baseline.)
Change Control. A process whereby modifications to documents, deliverables, or baselines associated
with the project are identified, documented, approved, or rejected.
Change Control Board. A formally chartered group responsible for reviewing, evaluating, approving,
delaying, or rejecting changes to the project, and for recording and communicating such decisions.
Change Control System. A set of procedures that describes how modifications to the project
deliverables and documentation are managed and controlled.
Change Request. A formal proposal to modify any document, deliverable, or baseline.
Code of Accounts. A numbering system used to uniquely identify each component of the work
breakdown structure.

Conceptual Design Phase. The first phase of the Design Stage, after passing the gate from the
Development Stage, that advances the definition of the scope and requirements, determines
feasibility, and produces updated drafts of most elements of the Project Execution Plan,
including parametric cost and schedule range estimates and a preliminary risk analysis.
Contingency. A planned amount of scope, budget, or time added to an estimate to allow for
items, conditions, or events for which the state, occurrence, or effect is uncertain and that
experience shows will likely result, in aggregate, in additional costs. Typically estimated using
statistical analysis or judgment based on past asset or project experience.
Contingency Report Table. A table containing a list of change control actions and allocations,
with ties to associated WBS elements and identified risk events, for all Performance
Measurement Baseline (PMB) changes that impact the use of contingency.
Constraint. A limiting factor that affects the execution of a project, program, portfolio, or process.

Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.

Section Revision:
May 2, 2016

8-3

Large Facilities Manual: NSF 16-0XX (May 2016)
8.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Construction Stage. The period of time in which funds are obligated for acquisition and/or
construction of a facility that fulfills the terms and conditions set forth in an award instrument
between NSF and the recipient(s). This stage ends with the start of the Operations Stage
Control Account. A management control point where scope, budget, actual cost, and schedule are
integrated and compared to earned value for performance measurement.

Corrective Action. An intentional activity that realigns the performance of the project work
with the project management plan.
Cost Book Report. A compilation of Cost Book Sheets, typically used to present total project
cost, but may be used to present rolled-up costs for smaller elements or sub-elements.
Cost Book Sheet. A compilation of related information from the Cost Model Data Set, used to
define and present the cost estimate for a particular element or sub-element of a deliverablebased work breakdown structure for construction or a functional (i.e., deliverable or activity
oriented) based work breakdown structure for operations.
Cost Estimating Plan. A plan describing how the cost estimating guidance in this manual will be
implemented, how the cost estimate will evolve over time, and how the “Cost Model Data Set”
will meet the various needs of the project. The CEP should typically include a narrative and
sufficient detail explaining the ground rules and assumptions, practices, systems, and
calculations used to develop the cost estimate.
Cost Model Data Set. The cost data used as input to software tools and/or project reports to
organize, correlate, and calculate different project management information.
Cost Performance Index. A measure of the cost efficiency of budgeted resources expressed as the
ratio of earned value to actual cost.
Cost Variance. The amount of budget deficit or surplus at a given point in time, expressed as the
difference between the earned value and the actual cost.
Crashing. A technique used to shorten the schedule duration for the least incremental cost by adding
resources.
Critical Chain Method. A schedule method that allows the project team to place buffers on any project
schedule path to account for limited resources and project uncertainties.
Critical Path. The sequence of activities that represents the longest path through a project, which
determines the shortest possible duration.
Critical Path Activity. Any activity on the critical path in a project schedule.

Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.

Section Revision:
May 2, 2016

8-4

Large Facilities Manual: NSF 16-0XX (May 2016)
8.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Critical Path Method. A method used to estimate the minimum project duration and determine the
amount of scheduling flexibility on the logical network paths within the schedule model.

Current Plan. The project cost and schedule plan reflecting the status of progress to date and
updated estimates for completing remaining work that is compared to the approved
Performance Measurement Baseline (PMB), as part of Earned Value Management.
Data Date. A point in time when the status of the project is recorded.
Decision Tree Analysis. A diagramming and calculation technique for evaluating the implications of a
chain of multiple options in the presence of uncertainty.
Decomposition. A technique used for dividing and subdividing the project scope and project
deliverables into smaller, more manageable parts.
Defect Repair. An intentional activity to modify a nonconforming product or product component.
Deliverable. Any unique and verifiable product, result, or capability to perform a service that is required
to be produced to complete a process, phase, or project.

De-Scoping Options (Plan). See Scope Contingency Plan.
Design Stage. The life cycle stage for detailed planning for projects approved by the NSF
Director at the end of the Development Stage and funded under the formal MREFC planning
process. It is divided into the Conceptual, Preliminary, and Final Design Phases; with a formal
and rigorous review gate at the end of each phase to show readiness for advancement to a
higher level of refinement with regard to scope, cost, and schedule.
Development Stage. The Facility Life Cycle stage in which initial high level ideas are developed
and a consensus built for the potential long-term need, priorities, and general requirements for
a large research facility of interest to NSF and the broader research community.
Discrete Effort. An activity that can be planned and measured and that yields a specific output. (Note.
Discrete effort is one of three earned value management [EVM] types of activities used to measure work
performance.)
Early Finish Date. In the critical path method, the earliest possible point in time when the uncompleted
portions of a schedule activity can finish based on the schedule network logic, the data date, and any
schedule constraints.
Early Start Date. In the critical path method, the earliest possible point in time when the uncompleted
portions of a schedule activity can start based on the schedule network logic, the data date, and any
schedule constraints.

Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.

Section Revision:
May 2, 2016

8-5

Large Facilities Manual: NSF 16-0XX (May 2016)
8.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Earned Value. The measure of work performed expressed in terms of the budget authorized for that
work.
Earned Value Management. A methodology that combines scope, schedule, and resource
measurements to assess project performance and progress.
Effort. The number of labor units required to complete a schedule activity or work breakdown structure
component, often expressed in hours, days, or weeks.
Enterprise Environmental Factors. Conditions, not under the immediate control of the team, that
influence, constrain, or direct the project, program, or portfolio.
*Estimate at Completion. The expected total cost of completing all work expressed as the sum of the
actual cost to date and the estimate to complete. (For NSF projects, contingency amounts are not
included in the ETC, EAC, BAC, or PMB due to the NSF requirement that contingency is held and
managed separately from the baseline.)

*Estimate to Complete. The expected cost to finish all the remaining project work. (For NSF projects,
contingency amounts are not included in the ETC, EAC, BAC, or PMB due to the NSF requirement that
contingency is held and managed separately from the baseline.)

Facility. Shared-use infrastructure, equipment, or instrument - or an integrated network and/or
collection of the same – that is either acquired or constructed to collect, analyze, and provide
necessary data and information in support of research having a major impact on a broad
segment of a scientific or engineering discipline.
Facility Life Cycle. The sequence of steps or stages that characterize the lifetime of a facility
from beginning to end. For NSF, the stages are Development, Design, Construction, Operations,
and Divestment.
Fast Tracking. A schedule compression technique in which activities or phases normally done in
sequence are performed in parallel for at least a portion of their duration.

Final Design Phase. The third and last phase of the Design Stage, after a successful Preliminary
Design Phase, that further refines the project Baseline Definition and the Project Execution Plan
and demonstrates that project planning and management meet requirements for readiness to
receive funding. The Final Design phase ends in a potential NSF approval to obligate
construction funds.
Finish-to-Finish. A logical relationship in which a successor activity cannot finish until a predecessor
activity has finished.
Finish-to-Start. A logical relationship in which a successor activity cannot start until a predecessor
activity has finished.

Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.

Section Revision:
May 2, 2016

8-6

Large Facilities Manual: NSF 16-0XX (May 2016)
8.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Forward Pass. A critical path method technique for calculating the early start and early finish dates by
working forward through the schedule model from the project start date or a given point in time.
Free Float. The amount of time that a schedule activity can be delayed without delaying the early start
date of any successor or violating a schedule constraint.
Gantt Chart. A bar chart of schedule information where activities are listed on the vertical axis, dates
are shown on the horizontal axis, and activity durations are shown as horizontal bars placed according to
start and finish dates.

Internal Management Plan. The internal document that defines NSF strategy for conducting
project oversight and assurance, managing NSF risk, and providing project funding.
Lag. The amount of time whereby a successor activity is required to be delayed with respect to a
predecessor activity.
Late Finish Date. In the critical path method, the latest possible point in time when the uncompleted
portions of a schedule activity can finish based on the schedule network logic, the project completion
date, and any schedule constraints.
Late Start Date. In the critical path method, the latest possible point in time when the uncompleted
portions of a schedule activity can start based on the schedule network logic, the project completion
date, and any schedule constraints.

Large Facility. A facility for which the construction cost is more than a specified percentage of
the sponsoring NSF organization’s budget plan, and which is paid for out of MREFC funds.
Construction costs for large facilities typically range between $100M and $800M.
Lead. The amount of time whereby a successor activity can be advanced with respect to a predecessor
activity.
Lessons Learned. The knowledge gained during a project which shows how project events were
addressed or should be addressed in the future for the purpose of improving future performance.
Level of Effort. An activity that does not produce definitive end products and is measured by the
passage of time. (Note. Level of effort is one of three earned value management [EVM] types of activities
used to measure work performance.)

Liens List. A list of expected adjustments to project scope, budget, and schedule contingency
amounts that are waiting for implementation, including formal change control actions for
planned baseline modifications, scope contingency options held for decision, and coverage of
variances.
Logical Relationship. A dependency between two activities or between an activity and a milestone.

Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.

Section Revision:
May 2, 2016

8-7

Large Facilities Manual: NSF 16-0XX (May 2016)
8.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Management. The act of controlling and making decisions about an operation, organization or
project; the act or process of deciding how to use something; the judicious use of means to
accomplish an end.
Management Reserve. A planned amount of money or time added to a baseline estimate to
address unforeseeable events (often referred to as “unknown unknowns”). Management
reserves are not allowable on NSF awards.
Milestone. A significant point or event in a project, program, or portfolio.
Most Likely Duration. An estimate of the most probable activity duration that takes into account all of
the known variables that could affect performance.

“No Cost Overrun” Policy. NSF policy requiring that a Total Project Cost estimate established at
the Preliminary Design Stage have adequate contingency to cover all foreseeable risks, and that
any cost increases not covered by contingency be accommodated by reductions in scope.
Operations Stage. The Life Cycle Stage that succeeds Construction and includes the day-to-day
work to operate and maintain the facility and to perform research. Operations may also include
activities to transition from construction to operations, replacement or upgrade activities,
technology research and development, and activities that support planning and staging for the
Divestment Stage.
Opportunity. A risk that would have a positive effect on one or more project objectives.
Optimistic Duration. An estimate of the shortest activity duration that takes into account all of the
known variables that could affect performance.
Organizational Breakdown Structure. A hierarchical representation of the project organization, which
illustrates the relationship between project activities and the organizational units that will perform those
activities.
Organizational Project Management Maturity. The level of an organization's ability to deliver the
desired strategic outcomes in a predictable, controllable, and reliable manner.

Oversight. Watchful and responsible care of something or some activity; regulatory supervision.
Parametric Estimating. An estimating technique in which an algorithm is used to calculate cost or
duration based on historical data and project parameters.
Path Convergence. A relationship in which a schedule activity has more than one predecessor.
Path Divergence. A relationship in which a schedule activity has more than one successor.

Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.

Section Revision:
May 2, 2016

8-8

Large Facilities Manual: NSF 16-0XX (May 2016)
8.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Percent Complete. An estimate expressed as a percent of the amount of work that has been completed
on an activity or a work breakdown structure component.

Performance Measurement Baseline. (Aka target baseline or performance baseline) The
approved cost and schedule plan for accomplishing project work scope that can be changed
only through formal change control process and that is used as a basis of comparison for
Earned Value Management. (For NSF projects, contingency amounts are not included in the
Earned Value tracking amounts due to the NSF requirement that contingency is held and
managed separately from the baseline.)
Pessimistic Duration. An estimate of the longest activity duration, which takes into account all of the
known variables that could affect performance.
Phase Gate. A review at the end of a phase in which a decision is made to continue to the next phase,
to continue with modification, or to end a project or program.
Planned Value. The authorized budget assigned to scheduled work.
Portfolio. Projects, programs, subportfolios, and operations managed as a group to achieve strategic
objectives.
Portfolio Management. The centralized management of one or more portfolios to achieve strategic
objectives.
Precedence Diagramming Method. A technique used for constructing a schedule model in which
activities are represented by nodes and are graphically linked by one or more logical relationships to
show the sequence in which the activities are to be performed.
Predecessor Activity. An activity that logically comes before a dependent activity in a schedule.

Preliminary Design Phase. The second phase of the Design Stage, after the Conceptual Design
Phase, that further advances the project definition and the Project Execution Plan. It produces a
bottom-up scope, cost, schedule, and risk analysis of sufficient maturity to allow determination
of the Project Total Cost and Duration for a stated future start date and to establish the MREFC
budget request.
Preventive Action. An intentional activity that ensures the future performance of the project work is
aligned with the project management plan.

Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.

Section Revision:
May 2, 2016

8-9

Large Facilities Manual: NSF 16-0XX (May 2016)
8.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Probabilistic Risk Assessment. A quantitative risk analysis that uses probability distributions to
represent the uncertainty usually present in the cost of a deliverable or the duration of a
scheduled activity, in order to obtain a range of outcomes for overall project cost and finish
dates that support selection of contingency amounts as part of risk management. Many
commercial probabilistic risk analysis applications employ Monte Carlo simulations of project
cost and schedule.
Probability and Impact Matrix. A grid for mapping the probability of each risk occurrence and its
impact on project objectives if that risk occurs.
Procurement Management Plan. A component of the project or program management plan that
describes how a team will acquire goods and services from outside of the performing organization.
Program. A group of related projects, subprograms, and program activities that are managed in a
coordinated way to obtain benefits not available from managing them individually.
Program Management. The application of knowledge, skills, tools, and techniques to a program to
meet the program requirements and to obtain benefits and control not available by managing projects
individually.
Progressive Elaboration. The iterative process of increasing the level of detail in a project
management plan as greater amounts of information and more accurate estimates become available.
Project Calendar. A calendar that identifies working days and shifts that are available for scheduled
activities.

Project End Date. The projected date for the completion of all the project baseline schedule
activities plus use of all schedule contingency. (Note that this date may be earlier than, but no
later than, the end date of the award instrument.)
*Project Execution Plan. The document that describes how the project will be executed, monitored,
and controlled.
Project Life Cycle. The series of phases that a project passes through from its initiation to its closure.
Project Management. The application of knowledge, skills, tools, and techniques to project activities to
meet the project requirements.
Project Management Office. A management structure that standardizes the project-related
governance processes and facilitates the sharing of resources, methodologies, tools, and techniques.
Project Manager. The person assigned by the performing organization to lead the team that is
responsible for achieving the project objectives.
Project Phase. A collection of logically related project activities that culminates in the completion of one
or more deliverables.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.

Section Revision:
May 2, 2016

8-10

Large Facilities Manual: NSF 16-0XX (May 2016)
8.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Project Schedule. An output of a schedule model that presents linked activities with planned dates,
durations, milestones, and resources.
Project Scope. The work performed to deliver a product, service, or result with the specified features
and functions.
Project Scope Statement. The description of the project scope, major deliverables, assumptions, and
constraints.
Quality Management Plan. A component of the project or program management plan that describes
how an organization's quality policies will be implemented.

Re-Baselining. Project re-planning that results in a change that is outside the terms set forth in
the award instrument for any of the following: 1) Total Project Cost (TPC); 2) overall project
duration or end date; or 3) project scope, except for approved options in the scope contingency
plan. Re-baselining actions require special review and approval by NSF beyond those of the
typical change control approval process for re-planning actions.
Re-Planning. A normal project management process to modify or re-organize the Performance
Measurement Baseline cost and/or schedule plans for future work without impacting total
project cost, project end date, or overall scope objectives; or the implementation of approved
de-scoping options. Formal change control processes are followed for all baseline changes.
Retroactive changes to past performance should not be included in re-planning.
Requirement. A condition or capability that is required to be present in a product, service, or result to
satisfy a contract or other formally imposed specification.
Resource Breakdown Structure. A hierarchical representation of resources by category and type.
Resource Calendar. A calendar that identifies the working days and shifts upon which each specific
resource is available.
Resource Leveling. A technique in which start and finish dates are adjusted based on resource
constraints with the goal of balancing demand for resources with the available supply.
Responsibility Assignment Matrix. A grid that shows the project resources assigned to each work
package.

Review and Recommend. The act of carefully looking at or examining the quality or condition
of something AND then suggesting that someone taken action or do something.
Risk. An uncertain event or condition that, if it occurs, has a positive or negative effect on one or more
project objectives.

Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.

Section Revision:
May 2, 2016

8-11

Large Facilities Manual: NSF 16-0XX (May 2016)
8.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Risk Acceptance. A risk response strategy whereby the project team decides to acknowledge the risk
and not take any action unless the risk occurs.
Risk Avoidance. A risk response strategy whereby the project team acts to eliminate the threat or
protect the project from its impact.
Risk Breakdown Structure. A hierarchical representation of risks that is organized according to risk
categories.
Risk Category. A group of potential causes of risk.

Risk Exposure. Quantitative impact of risk for a single event, quoted in currency or time, and
typically estimated from probability of occurrence and a likely impact or consequence. Overall
project risk exposure results from an accumulation of individual risk impacts for the work to be
completed, typically determined by applying probabilistic analysis to the set of individual risks.
Risk Management Plan. A component of the project, program, or portfolio management plan that
describes how risk management activities will be structured and performed.
Risk Mitigation. A risk response strategy whereby the project team acts to reduce the probability of
occurrence or impact of a risk.
Risk Register. A document in which the results of risk analysis and risk response planning are recorded.
Risk Transference. A risk response strategy whereby the project team shifts the impact of a threat to a
third party, together with ownership of the response.
Rolling Wave Planning. An iterative planning technique in which the work to be accomplished in the
near term is planned in detail, while the work in the future is planned at a higher level.
Schedule Compression. A technique used to shorten the schedule duration without reducing the
project scope.

Schedule Contingency. An amount added to a baseline schedule estimate to allow for identified
delays, conditions, or events for which the state, occurrence, or effect is uncertain and that
experience shows will likely result, in aggregate, in additional project duration. Typically
estimated using statistical analysis or judgment based on past asset or project experience.
Schedule Management Plan. A component of the project or program management plan that
establishes the criteria and the activities for developing, monitoring, and controlling the schedule.
Schedule Model. A representation of the plan for executing the project’s activities, including durations,
dependencies, and other planning information, used to produce a project schedule along with other
scheduling artifacts.

Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.

Section Revision:
May 2, 2016

8-12

Large Facilities Manual: NSF 16-0XX (May 2016)
8.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Schedule Performance Index. A measure of schedule efficiency expressed as the ratio of earned
value to planned value.
Schedule Variance. A measure of schedule performance expressed as the difference between the
earned value and the planned value.
Scope Baseline. The approved version of a scope statement, work breakdown structure (WBS) and its
associated WBS dictionary, which can be changed only through formal change control procedures and is
used as a basis for comparison.

Scope Contingency. Scope included in the project baseline definition that can be removed
without affecting the overall project’s objectives, but that may still have undesirable effects on
facility performance. Identified scope contingency should have a value equal to at least 10% of
the baseline budget.
Scope Contingency Plan. A component document of the Project Execution Plan that describes
how scope contingency is determined, monitored, and controlled over the project lifetime.
Scope Creep. The uncontrolled expansion to product or project scope without adjustments to time, cost,
and resources.
Scope Management Plan. A component of the project or program management plan that describes
how the scope will be defined, developed, monitored, controlled, and validated.
S-Curve Analysis. An earned value management technique used to indicate performance trends by
using a graph that displays cumulative costs over a specific time period.
Secondary Risk. A risk that arises as a direct result of implementing a risk response.
Sponsor. A person or group that provides resources and support for the project, program, or portfolio,
and is accountable for enabling success.
Staffing Management Plan. A component of the human resource plan that describes when and how
team members will be acquired and how long they will be needed.
Stakeholder. An individual, group, or organization that may affect, be affected by, or perceive itself to
be affected by a decision, activity, or outcome of a project, program, or portfolio.
Start-to-Finish. A logical relationship in which a successor activity cannot finish until a predecessor
activity has started.
Start-to-Start. A logical relationship in which a successor activity cannot start until a predecessor
activity has started.
Successor Activity. A dependent activity that logically comes after another activity in a schedule.
Summary Activity. A group of related schedule activities aggregated and displayed as a single activity.
Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.

Section Revision:
May 2, 2016

8-13

Large Facilities Manual: NSF 16-0XX (May 2016)
8.2 Terms and Definitions
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Divestment Stage. The stage in the facility life cycle encompasses divestment of the facility to
control by another entity or decommissioning, including dismantling/demolition, starting after
of the NSF Operations Stage ends and funding for divestment begins.
Threat. A risk that would have a negative effect on one or more project objectives.
Three-Point Estimate. A technique used to estimate cost or duration by applying an average or
weighted average of optimistic, pessimistic, and most likely estimates when there is uncertainty with the
individual activity estimates.
To-Complete Performance Index. A measure of the cost performance that is required to be achieved
with the remaining resources in order to meet a specified management goal, expressed as the ratio of
the cost to finish the outstanding work to the remaining budget.
Total Float. The amount of time that a schedule activity can be delayed or extended from its early start
date without delaying the project finish date or violating a schedule constraint.

Total Project Cost. The sum of the Performance Measurement Baseline budget and the budget
contingency.
Total Project Duration. The sum of the Performance Measurement Baseline schedule duration
and the schedule contingency.
Trigger Condition. An event or situation that indicates that a risk is about to occur.
Variance Analysis. A technique for determining the cause and degree of difference between the
baseline and actual performance.
Variance at Completion. A projection of the amount of budget deficit or surplus, expressed as the
difference between the budget at completion and the estimate at completion.
WBS Dictionary. A document that provides detailed deliverable, activity, and scheduling information
about each component in the work breakdown structure.
What-If Scenario Analysis. The process of evaluating scenarios in order to predict their effect on
project objectives.
Work Breakdown Structure. A hierarchical decomposition of the total scope of work to be carried out
by the project team to accomplish the project objectives and create the required deliverables.
Work Package. The work defined at the lowest level of the work breakdown structure for which cost
and duration can be estimated and managed.
Workaround. A response to a threat that has occurred, for which a prior response had not been
planned or was not effective.

Entries in italics are from Project Management Institute, Inc., [PMI Lexicon], (2012) Copyright and all rights reserved. Material
from this publication has been reproduced with the permission of PMI. Italicized entries with leading asterisks have been
modified slightly from the original PMI version for NSF purposes.

Section Revision:
May 2, 2016

8-14

Large Facilities Manual: NSF 16-0XX (May 2016)
Appendix A. Ranking Criteria for Prioritizing MREFC Projects
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

9

APPENDICES

Appendix A. Ranking Criteria for Prioritizing MREFC Projects
First Ranking: Scientific and Technical Criteria Assessed by Researchers in a Field or
Interdisciplinary Area (e.g., at the NSF Division level)





Which projects have the most scientific merit, potential and opportunities within a field
or interdisciplinary area?
Which projects are the most technologically ready?
Are the scientific credentials of the proposers of the highest rank?
Are the project-management capabilities of the proposal team of the highest quality?

Second Ranking: Agency Strategic Criteria Assessed across Related Fields (e.g., at the NSF
Directorate level)






Which projects will have the greatest impact on scientific advances in this set of related
fields taking into account the importance of balance among fields for NSF's portfolio
management in the nation's interest?
Which projects include opportunities to serve the needs of researchers from multiple
disciplines or the ability to facilitate interdisciplinary research?
Which projects have major commitments from other agencies or countries that should
be considered?
Which projects have the greatest potential for education and workforce development?
Which projects have the most readiness for further development and construction?

Third Ranking: National Criteria Assessed across All Fields (e.g., at the overall NSF level)







Which projects are in new and emerging fields that have the most potential to be
transformative? Which projects have the most potential to change how research is
conducted or to expand fundamental science and engineering frontiers?
Which projects have the greatest potential for maintaining US leadership in key science
and engineering fields?
Which projects produce the greatest benefits in numbers of researchers, educators and
students enabled?
Which projects most need to be undertaken in the near term? Which ones have the
most current windows of opportunity, pressing needs and international or interagency
commitments that should be met?
Which projects have the greatest degree of community support?
Which projects will have the greatest impact on scientific advances across fields taking
into account the importance of balance among fields for NSF's portfolio management in
the nation's interest?

Section Revision:
June 17, 2015

A-1

Large Facilities Manual: NSF 16-0XX (May 2016)
Appendix B. Memoranda Requesting NSB Approval of MREFC Project
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

Appendix B. Memoranda Requesting NSB Approval of MREFC Project
For MREFC Projects requesting NSB approval (for inclusion in a future budget request)
In addition to the NSF Form 10 (for clearance) and the Assistant Director/Office Head
endorsement(s), two items should be prepared and clearly marked as “pre-decisional – Do Not
Distribute.”
(1) A Director's Memorandum to Members of the NSB, briefly summarizing the project, the
need for the project and the Total Project Cost (TPC) Estimate and estimated duration. The
Director's Memorandum should include the following statement:
“With the Board’s concurrence that this project is meritorious and that its planning is
sufficiently advanced, the Director will take appropriate action in preparation of a
budget request. Board approval of this project for planning purposes does not imply
NSB approval of project implementation. Any such approval will be requested from the
NSB at the appropriate time.”
The Director's Memorandum should conclude with the following resolution:
“RESOLVED, that the National Science Board concurs that planning for the
is sufficiently advanced, and the intellectual value of the project sufficiently well
demonstrated, to justify consideration by the Director and the Board for funding in the
FY 20XX or a future NSF budget request.”
(2) A project report (usually six to eight pages) providing an update of the documentation
provided to the MREFC Panel.
For NSB approval of MREFC project implementation
Before project construction can be initiated, project implementation approval should be
granted by the National Science Board (NSB). First, the Director should prepare a Memorandum
for NSB Action.1 The Director's memorandum to the NSB should summarize information and
issues related to the proposed implementation of the project, potential policy
issues/implications, precedents involved, prior NSB discussion and any other factors that could
be considered non-routine.
It should normally contain: a brief science/engineering overview; a description of connections
to any national and international programs; a description of the project; a summary of the
review process and a short statement of response to any major concerns raised by reviewers; a
schedule with contingency; TPC which includes the performance baseline and contingency; the
impact that technological advances would have on the project during construction; the
1

See Proposal and Award Manual (PAM) VI.H.3.b.

Section Revision:
June 17, 2015

B-1

Large Facilities Manual: NSF 16-0XX (May 2016)
Appendix B. Memoranda Requesting NSB Approval of MREFC Project
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

percentage of program or division budgets that the proposed award represents and out-year
implications; and a description of plans for project management.
The Memorandum should also include a statement regarding plans for the end of the award
period, consistent with the policies set forth in NSB-08-16, “NSB Statement on Competition,
Recompetition and Renewal of NSF Awards” and the accompanying Resolution passed by the
NSB at its meeting of February 7, 2008.

Section Revision:
June 17, 2015

B-2

From:
To:
Subject:
Date:

Graff, Ivan
Wilkinson, Carol
LFM, minor addition: Section 3.4.1-10.2
Monday, May 16, 2016 1:15:54 PM

Greetings,
While meeting with Battelle two weeks ago it became apparent that brief description in LFM
Section 3.4.1, 10.2 EVMS, may or may not cover a description what ANSI/EIA 748C
describes in Section 3.7 as the “Earned Value Methodology” but others may describe as
“progressing rules.” Although one might argue that the word “processes” covers this, adding
“progressing rules” to the description may better highlight an area where the sponsor and
recipient need to come to some agreement. Please let me know if you think you could address
this in the LFM this time around if this requires wider discussion. Thanks!
Take care,

Ivan Graff, P.E., PMP, CCP
National Science Foundation
Budget, Finance, and Award Management (BFA)
Large Facilities Office (LFO)

4201 Wilson Boulevard
Arlington, VA 22230
Stafford II, Room 605.41

Email: [email protected]
Office: (703) 292-4416
Cell: (703) 859-2113
Fax: (703) 292-9005

NSF and LFO Improvement Suggestions
June 23, 2016
Note: Terms “facility,” “project,” “applicant,” “recipient,” “awardee,” and the like pertain solely to cooperative
agreements issued by NSF funding the acquisition, operation, or maintenance of property.

I.

Acquisition
A. Broadening Interest in Large Facilities
1. Require a solicitation, even for noncompetitive acquisitions, to provide a
public record of NSF requirements. For example questions that such a
solicitation might answer, see the historical "Guidelines for Development of
Internal Management Plans for Large Facilities" (c. 2006).
2. Develop a self-study review course on the LFM and compile job aids.
3. Prepare an information sheet identifying NSF community and funding
opportunities and resources for maturating a research infrastructure concept to
a completed facility.
B. Recipient Selection
1. Require relevant prior experience and prioritize qualified applicant entities
based on the nature of their past successes related to the scope they propose to
accomplish.
a. Recipients awarded funds for development need experience
developing design and construction planning documents.
b. Recipients awarded funds for design need experience preparing similar
designs and specifications.
c. Recipients awarded funds for construction need experience
constructing similar facilities.
d. Recipients awarded funds for operations need experience operating
similar facilities.
e. Recipients awarded funds for divestment need experience completing
similar cleanup, closure, and disposal actions.
2. Never assume that just one entity, especially an entity that submitted an
unsolicited proposal, has the capability and capacity to complete a scope of
work until market research (e.g., a request for letters of intent) shows
otherwise.

C. Ensuring Coordination
1. Require that an office of sponsored research or its equivalent review
applications and documents provided to external panelists, and indicate the
outcome of the review, prior to submission to NSF. NSF currently has such
controls in place for grants, but not yet for cooperative agreements.
D. Strategy
1. Discourage funding combinations of research and development, operations
and maintenance, and upgrades on the same awards to better track the kinds of
investments made and avoid reprioritizations without NSF input.
E. Evolving Policy
1. If NSF continues its current policy of holding recipients accountable to the
current version of the Large Facilities Manual (LFM), address in the LFM
how projects should request and document the need for equitable adjustments
due to NSF changes. Disallow management reserves from absorbing these
costs as NSF does not have an independent mechanism for estimating the
financial burden of its policy changes.
II.

Information
A. Project Documentation
1. LFM Section 4.4 lists some documents that projects share with NSF. Provide
a comprehensive listing and specify a flat organizational structure for filing
the documents to encourage each project to make retrieval as intuitive and
efficient as possible.
B. Software
1. Leverage Web 2.0 services endorsed by GSA to manage contacts (e.g.,
panelist and consultant inventory) and host discussion forums (e.g.,
communities of practice).
2. Obtain a site license (or about 30 individual licenses) for software that will
allow for reading and modest analysis of schedules prepared in Oracle
Primavera 6.
C. Information Collections
1. Customize application and reporting forms and templates to meet the
information needs of large cooperative agreements. Funding for large
IVAN GRAFF’S IMPROVEMENT SUGGESTIONS

Page 2

cooperative agreements comprise about 15% of NSF annual appropriations so
logically a commensurate portion of the content forms collect should serve
this subset of recipients.
2. LFO will need to work with DIAS Policy, IT services, and BFA and DAS
decision making bodies to facilitate this shift.
D. Financial Codes (Segment Values)
1. Add additional segments or options in existing segments to label funding
releases for contingency, management fee, and perhaps in the future
management reserve. NSF has struggled with tracking the amount of funding
released to a project earmarked for these two purposes.
III.

Policy
A. Large Facilities Manual
1. Reorganize for the primary audience
a. Recognize that recipients comprise the primary audience for the LFM.
Reorganize into mandatory and optional sections. Clearly distinguish
between requirements and commentary.
b. Remove content intended for NSF employees and relocate in standing
operating guidance (SOG). This includes references to documents
inaccessible outside of NSF such as the Proposal and Award Manual
(PAM).
c. Add an appendix listing in chronological order all instances where the
text directs recipients to obtain more complete direction from a
program officer.
2. Complete the Lifecycle
a. Require the submission of designs and specifications at each of the
three design phases. Describe the kinds of views and details that the
plans and specifications need to include.
b. Add a loop in the lifecycle process for a project to return to
preliminary design following delays in appropriations extending past a
year to allow the project to escalate and revalidate its cost estimate.
c. Introduce technical readiness levels for experimental systems and
equipment. Require documentation of increasing levels of maturity at
scales closer to the planned finished product through successive design
IVAN GRAFF’S IMPROVEMENT SUGGESTIONS

Page 3

reviews. The Construction Industry Institute offers a example of this
in its Project Development Rating Index (PDRI).
d. Discuss circumstances warranting, and procedures and requirements
for, rebaselining. Address the need, if any, to complete a change
request.
e. Add definitions for commissioning and transition to operations
(T2O). Add discussion comparing commissioning to transitioning to
operations.
i.

Commissioning: Base the definition on ASHRAE Standard
202.

ii.

T2O: Adopt the five step test used for NEON –
 Completion of construction
 Demonstration of functionality
 Appropriately staffed
 Budget allocated
 Segregation of funds controls in place

f. Require that operations awards specify performance measures and that
awardees report against them in annual reports.
g. Add stage gate reviews to enter and exit the divestment stage.
h. Reinforce the sequential nature of the stages and the independent
aspects of each stage.
3. Project Execution Plan (LFM Section 3.4.1)
a. Reorder the sections to follow a typical progression of project planning
and documentation preparation. Section 4 should begin with the
scope, then detail the schedule, and finish with documents that
contribute to and comprise the budget.
b. Add adequacy criteria for each section of the project execution plan
(PEP) organized by design phase and stage.
c. Add a section to the PEP for a communications plan that emphasizes
communications between NSF and the project. This plan would detail
a process and standards for issuing and resolving request for
information.
d. In Section 2.5, strike "outreach" since it appears earlier in Section 1.4.

IVAN GRAFF’S IMPROVEMENT SUGGESTIONS

Page 4

e. In Section 10.2, address "earned value methodology" [ANSI/EIA 748,
section 3.7]. For each WBS element, identify the ANSI/EIA option
most closely aligning with the described methodology: discrete effort,
apportioned effort, or level of effort.
f. Expand Section 11.1 to address existing conditions encountered by
upgrade projects since tying into or replacing existing systems poses
challenges similar to the constraints of a site or environmental
protections.
g. This section currently provides a brief description of each section.
Address each section more fully in dedicated subsequent chapters,
perhaps in the optional part of the LFM.
4. Costs
a. Address by reference (i.e., 2 CFR §200 and PAPPG) and by example
allowable and unallowable expenses.
b. Address spare supplies, material, and equipment including the
permissible –
i.
ii.
iii.

Source(s) of funds for their acquisition;
Classes or kinds; and,
Quantities or volumes.

c. Provide on the public LFO Website templates for documenting and
tabulating cost estimates.
5. Panel Reviews
a. Describe the similarities and differences between all external panel
reviews convened during a facility’s life cycle with an emphasis on the
responsibilities of, and impacts to, the recipient.
b. Add lists of documents (1) the recipient must supply and (2) the panel
must receive for each phase and stage review.
c. Give panelists access to past deliverables, monthly and annual reports
from the project, and relevant portions of bi-monthly reports to the
Director where available to better contextualize the project.
d. Ensure the project sets up and maintains systems scrutinized in a
business systems review (BSR).

IVAN GRAFF’S IMPROVEMENT SUGGESTIONS

Page 5

e. Require the program officer (PO) to issue direction to the project
based on the comments and recommendations of a review panel, rather
than pass along the report with an expectation that the project will
effectively prioritize and implement it. Note that the review panel
SOG(s) will explain that the PO has separate relationships with the
panel and the project while the panel and the project have not entered
into an agreement.
6. Oversight
a. Execution of the project must include implementing accountability
practices, procedures, and systems that a BSR will confirm. To help
projects comply with the BSR Guide, repeat the table appearing in
Section 3.4.1 (PEP), delete the description column, and add a column
for the "organization process assets" (PMI term), i.e., the components
of a BSR, on which the activities comprising that portion of the PEP
depend.
b. Require a successful BSR prior to administratively closing out a
construction cooperative agreement.
7. Reporting
a. Detail requirements for preparing monthly and annual reports,
especially during the design and construction stages.
b. Have the program officer send a response to monthly reports to the
project:
i.

Prepare a template that would replace or supplement section 6
of the monthly report that would indicate:
 Errors and omissions;
 Positive and negative impressions;
 Corrective expectations for the next month or later; and,
 An overall confidence level for the future of the project.

ii.

Inform this feedback with input from members of the
integrated project team (IPT).

iii.

LFO would use the whole or portions of the section 6 feedback
verbatim in its bimonthly report to the Director.

c. Introduce an annual report format for facilities in operations.

IVAN GRAFF’S IMPROVEMENT SUGGESTIONS

Page 6

8. Guidance Chapters
a. Add a chapter for work breakdown structures (WBS) and the WBS
dictionary.
i.

Differentiate through example project deliverables or products
from process activities or administrative requirements.

ii.

Differentiate between conventional and experimental facilities

iii.

Include an element in the WBS for each unique item of
significant property.

b. Reference environmental reviews and compliance in conjunction with
development and conceptual design. In the course of doing so:
i.

Revise content to convey that other kinds of environmental
reviews besides those currently described may have to occur;

ii.

Strike references to internal guidance; and,

iii.

Clarify that environmental compliance remains the
responsibility of NSF while the project must obtain permits and
licenses.

B. Internal Procedures
1. Assign the responsibility of writing and managing financial policy to an office
within BFA and provide it with the resources (staff, budget, and authority) it
needs.
2. Management Reserves
a. Future MREFC appropriations requests may include more than the
total project cost (TPC). The TPC includes the performance
management baseline, management fee, and contingency, but does not
include funding to cover NSF risks.
b. Allow management reserves for:
i.

Scope contingency detailed by the project, desired by NSF, but
at risk of getting descoped due to draws on contingency; or,

ii.

Work packages executed by NSF in support of the cooperative
agreement such as environmental impact statements.

IVAN GRAFF’S IMPROVEMENT SUGGESTIONS

Page 7

c. NSF must base its request for management reserves on sound,
verifiable cost estimates obtained prior to entry into final design.
d. Do not use management reserves to augment a TPC to compensate for
deficiencies in a proposal from the project or other set of project
documents. Instead, encourage the project to resolve the deficiencies
and then proceed for a review of updated documents.
3. Revise and restructure the approach LFO uses for writing and revising policy.
a. Select an LFO analyst who will write and revise the SOG and bear the
responsibility for addressing all comments received, regardless of their
source.
b. Develop a comment tracking template to serve as a record of the
evolution of a SOG.
c. Notify the LFWG membership of the scope of the SOG or SOG
revision and compile feedback.

d. The analyst will facilitate a series of review sessions scheduled for one
week following when invited attendees’ will receive a review draft.
e. Hold the following series of reviews:
Audience
LFWG
LFO
BFA
LFWG (virtual)
LFO
LFWG (in person)
SMaRT (if needed)

Type of Review
Notification (email)
Draft #1
Draft #2
Draft #3
Draft #4
Draft #5
Final

Timeline
Week 1
Week 4
Week 6
Week 8
Week 10
Week 12
Week 14

f. Only the assigned LFO analyst may delay the progression of drafts.
DIAS will issue the SOG or SOG revision by Week 16.
g. LFO will commit to review and revise as necessary each SOG at least
once every three years.
4. Standing Operating Guidance (SOG)
a. Ensure that for all instances in the LFM where the text directs
recipients to obtain more complete direction from a program officer, a
SOG addresses that direction.
IVAN GRAFF’S IMPROVEMENT SUGGESTIONS

Page 8

b. Consolidate the planned array of SOGs as of May 2016 into as few
documents as possible as the current list has grown and the central
themes of information collection, reviews, and assurance have
emerged. The current array includes:

i.
ii.
iii.
iv.
v.
vi.
vii.
viii.
ix.
x.
xi.
xii.
xiii.
xiv.

MREFC Panel
Environmental Compliance
Internal Management Plans
Integrated Project Teams
Cost Reviews
eJacket
Monthly Construction Reports
Contingency
Design Stage Review
EVM Guide and Validation
Construction Stage Review
Property Reporting
Annual Planning Process for Business Systems Reviews
Scoping Business System Reviews

c. Additional topics that one or more SOGs need to address (suggested
additional topics appear in bold)
i.
ii.
iii.
iv.
v.
vi.
vii.
viii.
ix.
x.
xi.
xii.
xiii.
xiv.
xv.
xvi.
xvii.
xviii.

LFO SOG writing and updating procedures and
assignments
Document Organization
Solicitations
MREFC Panel
Environmental Compliance
Internal Management Plans
Integrated Project Teams
Cost Reviews
Assessing the Adequacy of a Project Execution Plan and its
Subsidiary Plans and Documents
Cooperative Agreements and Amendments
eJacket
Funding Releases and Accounting
Monthly and Annual Design and Construction Reports and
Bimonthly Reports to the Director
Change Request Reviews
Contingency
Rebaselining
Design Stage Review
EVM Guide and Validation
IVAN GRAFF’S IMPROVEMENT SUGGESTIONS

Page 9

xix.
xx.
xxi.
xxii.
xxiii.
xxiv.
xxv.
xxvi.

Construction Stage Review
Property Reporting
Assessing the Adequacy of an Operations Plan
Annual Planning Process for Business Systems Reviews
Scoping Business System Reviews
Annual Operations Reports
Operations Stage Review
Divestment

d. Include in the design and construction reports SOG language that the
program officer will share his or her feedback with the project and that
the same feedback will appear in the LFO’s bimonthly report to the
Director.
e. Combine all external review related SOGs and all internal review
related SOGs into a single external review SOG and a single internal
review SOG.

Stay in touch:
http://www.energy.gov/phonebook/Ivan%20Graff

IVAN GRAFF’S IMPROVEMENT SUGGESTIONS

Page 10

From:
To:
Cc:
Subject:
Date:

Hawkins, Matthew J.
Wilkinson, Carol
Porter, Kevin
Re: Contingency for MREFC Fixed Price Contracts
Wednesday, August 10, 2016 10:32:13 AM

Correct on UG and profit vs non-profit....the recipient being for-profit or non-profit is what
Bart said earlier and is covered by the existing footnote.
Contingency is allowed under UG.  "Reserve" or "contingency reserve" is not...see SOG 20162 where we cover these definitions as well.  Perhaps these definitions should also be included
in Section 8.2 (Lexicon)?
Matt
Sent from my iPad
On Aug 10, 2016, at 10:20 AM, Wilkinson, Carol wrote:
Oops. The OMB circular has been replaced by the uniform guidance, so that covers
contracts with non-profits and educational institutions.

So we will have to use both the FAR and the OMB Uniform Guidance for contracts,
depending upon the type of institution.

Carol

Carol Wilkinson
National Science Foundation
Large Facilities Office (LFO) 
4201 Wilson Boulevard
Arlington, VA 22230

Office: (703) 292-8335
Cell: (509) 378-5431

From: Hawkins, Matthew J.
Sent: Tuesday, August 09, 2016 2:43 PM
To: Wilkinson, Carol
Cc: Porter, Kevin
Subject: Re: Contingency for MREFC Fixed Price Contracts

Carol,

I think we need some very simple "applicability" statements in the opening sections of
the LFM.

"Unless otherwise noted, the policies and procedures in the manual apply to research
infrastructure projects regardless of the award instrument employed.  Cooperative
agreements are governed by the Uniform Guidance.  Contracts are governed by the
FAR."

I know how I handled contingency with FFP sub-contracts under CA's.  

What does FAR say about budget contingency?

Matt
Sent from my iPad
On Aug 9, 2016, at 2:08 PM, Wilkinson, Carol wrote:
Matt,

The LFM is not clear about contingency expectations in the case of
contracts. How do we handle the contingency issue with respect to full
fixed price contracts instead of CAs– including return of contingency at
the end? I also need to know if the EVM reporting for FFPCs has
contingency in the baseline for EVM reporting or if it is carried separately
as it is for CAs. On funding, I don’t think we can hold back ‘contingency’
for a contract like we do for CAs.

I’m not clear on the differences between contracts and CAs and neither is
the LFM, as the AIMS folks have pointed out.

Carol

Carol Wilkinson
National Science Foundation
Large Facilities Office (LFO) 
4201 Wilson Boulevard
Arlington, VA 22230

Office: (703) 292-8335
Cell: (509) 378-5431

From:
To:
Cc:
Subject:
Date:

Graff, Ivan
Wilkinson, Carol
Hawkins, Matthew J.
LFM Errata: Commissioning Plan and "IMP"
Wednesday, May 11, 2016 2:15:40 PM

Good afternoon,

In case we have an opportunity to correct what seems like a typographical error in the LFM . .
..

In LFM (15-89) Section 2.4.2, Commissioning Plan, in the second sentence which reads “The
scope of commissioning work is to undertake initial operation of the facility and bring it up to
the design level of operation in accordance with the IMP,” I suggest striking “IMP” and
replacing it with “PEP” or if we have the latitude to revise the entire sentence, replace it with:
“Commissioning verifies that the substantially complete facility operates over its full range of
capabilities as specified in final design documents.”

Take care,

Ivan Graff, P.E., PMP, CCP
National Science Foundation
Budget, Finance, and Award Management (BFA)
Large Facilities Office (LFO)

4201 Wilson Boulevard
Arlington, VA 22230
Stafford II, Room 605.41

Email: [email protected]
Office: (703) 292-4416
Cell: (703) 859-2113
Fax: (703) 292-9005

Comments on Large Facilities Manual – Jean McGovern

Section/Paragraph

Comment

2.1.3-1, last
paragraph

Who are the individuals in the following statement?
Individuals should discuss any proposed departures with the cognizant
Program Officer.
Note: If this is an internal document, are individuals from the Awardee
permitted to provide input?
IMP should be defined prior to this section as the reader does not have context
to the paragraph meaning. ( It is defined in the later sections of the document )
I think the document could use a description of science planning, academy
reports, Advisory Committee comments and how they shape facilities at NSF in
the introductory section (pull up language from section 3.3)
Can industry lead facility projects or only non-profits/academic institutions?
(This is usually an FAQ, so perhaps addressing it upfront in the summary section
would be beneficial)
NEPA and permitting responsibilities (not just environmental impacts) should be
articulated in the chart. Does one need the EA or EIS prior to construction
funding request? Editorial Note – label Figures on bottom, Tables on top
What does industrialize key technologies mean? (not a modern term)
When is the Conops delivered and reviewed?
Property management/ownership model should be included or in the PEP. This
is very important for developing the design. (lease, NSF own, Awardee own,
there are a lot of options to trade off and assess)
What is the systems engineering requirement for Large Facility Projects?
What is the output or definition of a developed Project Controls System?
This to be written Section 2.1.5 should include internal competition planning
text as part of this section.
The roles and responsibilities section contains a lot of verbs. In many cases the
deliverables and sign off associated with these verbs are unclear. For example,
what does assurance deliver and how is it delivered? (a report, a letter, email,
witness, controlled document, etc) This section would be a lot stronger if the
deliverables and location of the filing of these deliverables would be defined.
Appendix B only has some of the information.
The statement “The HLFO develops and implements processes for insuring
that all facility award instruments” is unclear. What are the processes, how
are they implemented, who delivers and when? Is it a report, email,
discussion? This is unclear.
I think the paragraph below requires definition of the deliverable that
describes the assurance or certification of the total project cost of an MREFC
or Operations Project. The paragraph below is unclear. Is it a letter to the
CFO certifying the TPC? What gets delivered? I think this is the most
important authority that HFLO has in the document. Who is responsible for
the project independent cost estimate? Is it LFO or the Directorate?

2.1.3-1, last
paragraph
General
Section 2.1.4
Summary of
MREFC
Figure 2.1.4-1
Figure 2.1.4-1
Figure 2.1.4-1
Figure 2.1.4-1
Figure 2.1.4-1
Figure 2.1.4-1
Section 2.1.5
Section 2.1.6

HLFO Section

HFLO Section

HFLO Section –
monthly report

Section 2.1.6.3 IPT
Section 2.1.6.4
MREFC Panel &
DRB

Section 2.2.2
Approvals
Section 2.3.3.1
Bullet 2,
requirements

“Prior to NSF requesting NSB approval to include a proposed project in a
future budget request, the HLFO contributes to agency assurance that the
project plans are construction ready, and that the construction and
operations budgets are satisfactorily justified.1 This assurance comes through
assignment of the LFO Liaison to the IPT and membership (as assigned) on
various governance bodies such as the Director’s Review Board and MREFC
Panel.”
What are the specific monthly report requirements for the PO? These
requirements have changed over time, but are not specified in the manual.
In the past, the requirement was to submit the awardee’s submission that
met LFO requirements with a summary from the PO. This was not uniformly
applied as some POs wrote their own reports based on the Awardee
submissions. The report sections for an MREFC project would benefit from
standardization as the Awardees could then use the Manual for direction.
Some CAs have clauses written to specify reports, so maybe the Manual
should direct the Awardee to the award clause.
Minutes of the meeting should be required and communicated to assurance
functions.
What are the deliverables from the MREFC panel with respect to assurance
and prioritization? Is the facilities plan or budget request timeline prioritized
by the MREFC panel? Is there a charter for the MREFC panel that describes
functioning and quorum voting, etc? I think the information in Appendix B
may not suffice.
What is the format for the approvals noted in this section? (letter, email, as
described by another policy, etc) Appendix B has some information, but not
all.
The Manual really needs a systems engineering description of this bullet and
the relationship to Conops, requirements, derivation, acceptance (test,
integration, verification, validation, commissioning and Operational
Definition. Here is an example of what is needed:

The manual should utilize some sort of INCOSE process and break down the
process to address the development, construction and transition processes
more clearly than “functional requirements”.
Given the unique nature of NSF projects, I do agree that in the conceptual
stage there needs to be traceability from the science question(s) or science
to the infrastructure conceptual design. In some cases, I think NSF should
think about getting external technical review on the traceability before
starting the investment in projectizing as this is very expensive and can cause
a project to develop without the proper scientific input and prioritization. I
would also recommend that projects get a technology readiness level
assessment at the traceability (pre-concept design review stage) and get
scientific input/external review at that point as well. The definitions of
“industrialization, non recurring engineering, subsystem development and
their relation to technology and engineering readiness need to be included in
the Large Facilities Manual more explicitly.

2.3.1.1

2.3.2 PDR phase
2.3.2

2.3.2
2.3.2
2.3.2

2.3.3.2

I recommend adaptation of INCOSE processes into facility guidance and the
Manual. This is a lesson learned at NSF that needs to be incorporated into
the Manual from multiple projects.
This section should include a concept of operations. The basics of how the
facility is envisioned to be operated and the user requirements as well as the
system requirements. This is more specific and a necessary addition as it
serves to start user/operator requirements that could be different than a
“community outreach” section of the PEP. The User requirements are often
omitted from the system requirements or science objectives and missed.
This is especially important for cyberinfrastructure projects. (basic systems
engineering – above)
When do the operations costs or ceilings get defined? How are they
developed and iterated?
How does the no cost overrun policy apply to operational costs? If the ops
budget changes, what is the process to right size the scope to assure that the
facility being built can be operated to cost? This is a lessons learned from
past projects. How is an operational budget change communicated to the
MREFC panel & DRB post-approval?
The design process does not include text on design/build vs design/bid/build
and how this is handled with respect to project readiness.
NEPA -- Is the FNSI needed or just commencement of the process? In some
cases permits involve a lot of risk. How should this be addressed in the text?
What are the requirements for operations budget reviews during the
construction phase? How does NSF assure that cost growth is managed
along with scope and TPC?
“Fit up” is not a modern term. Use an INCOSE lexicon for awardees
(interfaces, ICDs, ICAs, etc). This is a lessons learned. Again, the terminology

2.3.3.2 FDR

2.3.3.3

2.4.1

2.4.1

2.4.2

2.4.3

2.5.1

2.5.1

with respect to systems engineering, requirements, IVT and commissioning is
not modern or to generally accepted engineering practices. This is a lesson
learned and has been an FAQ at many design reviews.
Quality and Safety Management should be clearly articulated at FDR and
integrated into schedule and cost baselines. Change the wording of “plans”
to management and staffing policies, procedures and staff. This is a lesson
learned.
Awardees are responsible for conducting cost analysis and delivering it to
NSF for review. At what step in the process is the cost analysis from the
Awardee delivered and reviewed? It should be delivered with every cost
estimate at every design review. Where is the cost analysis standard for the
Awardee provided in the Manual?
I think there should be more language to describe the baseline maintenance
process we expect during construction. The replan definition seems to allow
Awardees to allocate schedule contingency without tracking variances to
original plans. This is also an EVM comment. I think more guidance is
needed on this in the Manual.
Is the OMB requirement to report any impact to a science requirement in the
monthly report still valid? How is scope management (descoping) included
in the baseline maintenance section?
The Commissioning section does not use modern engineering language. I
recommend incorporating INCOSE language and aligning the commissioning
strategy with the acquisition strategy. The commissioning plan should be as
detailed as the Final Design and acquisition strategy allows. If the plan is
allowed to float to year 5 of a 6 year project, it will not have the proper
upfront time and budget allocation. Lesson learned on many NSF projects.
Too often weak definition of upfront IVT/commissioning results in transfer of
costs and risk to the O&M phase. The year 1 O&M budget needs to be
clearly reviewed based on the IVT/commissioning at FDR. Year 1 ops
budgets should be higher than later years as this is a best practice. In some
cases Year 1 should be 20-30% higher than Year 2.
Project Close out costs must be part of the TPC. Therefore this section needs
to be completed and the requirements of this section should be included as
sections in the PDR and FDR sections. Is project close out an MREFC or R&RA
cost? This interpretation should be in the policy.
There is standard award language that indicates the Program Officer
provides the Awardee with Annual Work Plan and Annual Report Guidance. I
think these sections should perhaps reference the Awards as the language
provided in these sections provides a perception that the AWP and Annual
Report are “lightweight” as compared to the PEP. In some cases, the AWP
will have a framework that is similar to the PEP and incorporate more user
and science engagement.
Ops Reviews should review the Operations Plan or Manual that the MREFC
effort produced. It might also be helpful to review GPRA at the review and
additional metrics for facility performance that should be established via a

2.5.2

2.6

3.4.1-1

3.4.1-1
3.4.1-1
3.4.1-1

3.4.1-1
3.4.1-1

3.4.1-1
3.4.1-1
3.4.1-1
3.4.1-1

IEEE Conops (if systems engineering language is incorporated). This is not
burdensome, it is necessary for the management to have a set of deeper
metrics than GPRA to self-assess facility operations and conduct prudent
management.
The NSB Recompetition statements and policies should be cross referenced
in this section. The IMP should be updated at the recompete stage and
included in this section.
Divestment is covered in two places 2.6 and 3.1.1. The discussion on how to
manage divestment prior to defining divestment is awkward in the Manual,
especially since in section 3.1.1 you state it as undefined. I recommend in
this version that you just the TBD in 2.6 as well so that the reader does not
have to search ahead for no definition. Since the divestment strategy
influences construction strategy, I recommend getting it defined and
reviewed up front. For example modularization, segmentation and
incremental build might be a necessary construction strategy based on
divestment (LIGO I and LIGO II, etc).
The Project Development Plan elements should be made more clear as they
are foundational to cost estimation. The CEP development must be in the
PDP
The PDP should include a section on systems engineering and SEMP
The PEP or PDP should include the SOP requirements from the NEPA
documentation. These are sometimes expensive cost drivers
The Community Outreach should include the SOPs created by the
environmental documents and a section on media relations. There are
standard processes used for large construction efforts across NSF.
Sections 7 and 15 are really the same – see the systems engineering “V” above. I
recommend leaving Operations in 15 and moving the IVT to Systems
Engineering.
Section 5 – Staffing – some projects may require an indirect review or Staffing
Integration management review. I think it is important to highlight the
integration between back and front offices and assure that the proper direct and
indirect support are clearly articulated. This goes to a description of what each
indirect rate applies at each institution. This will assure that costs assumed as
covered by indirect are actually covered and that the project, NSF and reviewers
are clear on cost allocations. Sometimes what is in the rate, does not get
applied in practice and this should be resolved for projects up front.
I recommend that section 13 be named Environment, Safety and Health (ES&H).
NEPA is a continuous process and needs to be managed during construction with
safety and health. Usually they are in one scope….after siting….
16.3 – Conops is part of the PDP and systems engineering effort, not part of
project close out.
Divestment is a construction requirement and needs to be in the PDP/systems
engineering sections (up front, like conops)
The PEP should have a section on deliverables. This should include: 1.
Technical Data Package (standards, plans, specs, documentation) 2. Manuals for
Operations (Operations Manuals) 3. Standard Operating Procedures 4.

3.4.1-1

3.5
3.5
3.5

3.5
3.5.2
General
4.2.2.4 & General
4.2.2.3 Cost report
4.2.3.3

General
4.2.3.3
4.2.3.4
4.2.5-1 Change
Control

Cybersecurity and COOP 5. Data specifications and data lifecycle management
specs/procedures. The deliverable section should also assure the delivery
medium (CDs, paper, website) are specified. The MREFC and R&RA acquisition
strategy maps to this statement as the TDPs and Ops Manuals will be different
for a turn key vs deliver only. This section should form the basis for funding
allocation and auditing.
Change from divestment to downsizing, decommissioning and/or divestment.
Request that appropriate high level plans, design requirements and
schedule/cost are attached to the strategy. What happens if NSF needs to cut
operations costs and downsize during the construction period? What are the
mid-construction off ramps to address this situation?
Replace IT security with cybersecurity
Add data management plan in place as an assessment.
This section is “light” on proposal preparation requirements. If NSF requires
that proposals meet GAO cost requirements, then this section needs to
encompass the GAO requirements (WBS, set of requirements, etc, etc). At first
pass, the Ops Plan would need to have the same requirements as the PEP for
MREFC.
I suggest you change the title of this section to Management & Operations or
Operations & Management. This is the conventional term.
Does the recomp guidance map to the NSB guidance provided in November
2015?
Provide the location of the internal guidance documents
What is the hierarchy of compliance for documentation? Is it the supplemental
info in this Manual or the PAPPG? Shouldn’t the Awards Specialist be the person
who is providing guidance on this matter (maybe via the Program Officer)?
I recommend having the report signed by the AOR and PI prior to submission. I
also recommend that the Awardee Cost Analysis of the construction report be
an appendix of the cost report
Emphasize that the WBS should be deliverable based. Emphasize activity based
estimates versus LOE. To me, it looks like you are encouraging all IVT to be LOE.
I think a statement on minimizing all LOE where practicable would be good
guidance.
Where is the Annual Work Plan guidance for MREFC projects? How does the
AWP for MREFC map to the baseline maintenance processes?
Define integration, verification, validation and test. Recommend that this get
done at subsystem, system and system of systems levels.
Is BOE refinement the Awardee Cost Analysis? It seems confusing.
There should be a from/to table for rebudgeting and this should be encouraged
in the Project Controls Implementation section (or designed). The table on the
bottom of the sample form may not allow for clear traceability across accounts,
only drawdown. The process for indirect allocation and financial officer sign off
(prior to NSF sign off) should be included. This is a lesson learned. Indirect
allocation from contingency can then get tracked as well (with proper from/to
table)

4.2.5.8
4.2.5.9
4.3

4.4

4.4.2-1 figure (5)
General

4.4.2-1
4.4.2-1

4.6
General
5.2

Environmental

I think putting the “second level” WBS requirement may skew the scope
delineation. I recommend that the Awardee propose a reporting level and the
panels/NSF decide on the proper reporting.
What about in-kind contributions? How should these be handled?
Use INCOSE language to describe systems engineering and a “V” diagram. The
SEMP should have an outline or set of criteria. The acceptance in the Quality
program or subcontracts and acceptance at the system/subsystem performance
should be clear. (acceptance to spec, acceptance to performance requirements
with defined levels (user, witness, cert, etc)
There are many documents that are missing from this section. Go through the
PEP, add the Operations Manual and develop a list of what is required to
recompete the facility. This will provide the comprehensive document list NSF
needs. Plans, specs, SOPs, workflows, QA/QC, safety, cybersecurity, data
management and many more. IF NSF needs to recompete, what is the
acceptable quality level for the documentation and how do the documents get
delivered?
Reference the definition of Operations Contingency in the figure. In the section,
hotlink the Uniform Guidance
I think there needs to be information on rebudget authority as a method of
management prior to contingency allocation in an operations or MREFC effort.
Rebudgeting may be different than descope as it provides the opportunity to
continuously assess time phased scope priorities. For operations this is critical
and it should be assessed first, prior to any contingency requests. Rebudgeting
may require deobligation by the Awardee and assuring that there is an
acquisition strategy/contract to support this method is important.
Education & Outreach budgets (EHR vs directorates). I recommend putting (if
allowable) next to this term
This figure seems like it is encouraging LOE for Operations. I think there should
be guidance on encouraging activity based estimates where practicable. This is a
standard M&O panel recommendation and should be incorporated into the
Manual
Is there a definition for MOA? Sometimes MOUs are between the Awardee and
foreign countries, this section indicates only NSF can have MOUs… I think it
depends on the ownership model and other factors
Where is the guidance for the Awardee Advisory Committees for the project?
This is a lesson learned for large subawards/subcontracts.
Because expert judgement is required for risk assessment, I recommend that the
Awardee assure the experts are vetted by their project advisory committee and
maybe even NSF prior to conducting the assessment. This should be a
recommendation and not a requirement
Where are the decision milestones and off ramps defined? Does a PEA need to
be completed by PDR and noticed? This is a lesson learned from past projects.

From:
To:
Subject:
Date:
Attachments:

Plimpton, Suzanne H.
Wilkinson, Carol; Hawkins, Matthew J.; Whitehurst, Eddie
FW: "For comment" draft of the Large Facilities Manual (LFM) published in the Federal Register
Friday, July 08, 2016 12:05:31 PM
image001.png

Here’s OIA’s comment.

From: Frye, Joan M.
Sent: Thursday, July 07, 2016 9:33 AM
To: Plimpton, Suzanne H.
Cc: Iacono, Suzanne ; Frye, Joan M.
Subject: FW: "For comment" draft of the Large Facilities Manual (LFM) published in the Federal
Register

Hi Suzanne,

No comments from OIA.

Thanks,

Joan

Joan M. Frye, PhD
Acting Deputy Office Head
Office of Integrative Activities
National Science Foundation
4201 Wilson Boulevard, Suite 935
Arlington, VA  22230

From: "Frye, Joan M."
Date: Monday, May 16, 2016 at 10:52 AM
To: OIA ALL
Subject: FW: "For comment" draft of the Large Facilities Manual (LFM) published in the Federal
Register

Good morning!

Please provide comments to me by cob July 6th, so that I can compile and send to Suzanne by the deadline.

Thanks,

Joan

Joan M. Frye, PhD
Senior Staff Associate
Office of Integrative Activities
National Science Foundation
4201 Wilson Boulevard, Suite 935
Arlington, VA  22230

From: "Hawkins, Matthew J."
Date: Monday, May 16, 2016 at 10:20 AM
To: NSF SMART , NSF DADOH , NSF Division Directors
, NSF Program Officers , NSF AMG , NSF AO/OS
, NSF Deputy Division Directors , BFA ALL ,
OGC All , DIS ALL , DAS Staff , BFA BPLG

Cc: NSF POLICY , "Hawkins, Matthew J."
Subject: "For comment" draft of the Large Facilities Manual (LFM) published in the Federal Register

Dear Colleagues:

On May 9th, NSF published a notice in the Federal Register announcing the availability of a “For
comment” draft of the Large Facilities Manual (LFM).   We are soliciting Directorate/Office
comments on the draft LFM until COB July 8, 2016.   Please note that all comments should be
coordinated through the applicable Assistant Director/Office Head’s office and sent to  Suzanne
Plimpton at [email protected].

The following are links to the Federal Register Notice and the draft LFM:

· https://www.gpo.gov/fdsys/pkg/FR-2016-05-09/pdf/2016-10793.pdf
· http://www.nsf.gov/bfa/lfo/lfo_documents.jsp

Any technical questions should be directed to Dr. Carol Wilkinson in the Large Facilities Office at
[email protected].

Kind Regards,
Matt Hawkins

nsflogo

Matthew Hawkins

Head, Large Facilities Office
Office of Budget, Finance & Award Management
voice: x7407
email:  [email protected]

National Science Foundation • Office oflnspector General
4201 Wilson Boulevard, Suite I-1135, Arlington, Virginia 22230

MEMORANDUM
DATE:
TO:

FROM:

JUL 8 ~· 2ill6
Suzanne Plimpton
Reports Clearance Officer, OGC

~~~~

Assistant Inspector General for Audit (Acting)

SUBJECT:

Comments on the National Science Foundation's Large Facilities Financial
Data Collection Tool

We are submitting the following comments on the National Science Foundation's (NSF) draft
Large Facilities Financial Data Collection Tool (reporting tool) for NSF's consideration. We
understand that the overall purpose of the reporting tool is to provide for the collection of cost data
from each awardee for each large facility, to increase NSF's management and financial oversight
of its large research facilities. Collection of this information will provide NSF valuable budget
and expenditure information needed for NSF's oversight purposes. However, we note overall that
the reporting tool is not the equivalent of an incurred cost submission (which, among other things,
reports total recipient direct costs for all of a recipient's awards and calculates and proposes
indirect cost rates subject to negotiation). Therefore, in terms of facilitating an incurred cost audit,
it seems that the reporting tool, at best, could only facilitate a direct cost audit with a review of the
application of indirect cost rates of the award being reported. NSF would have to further
coordinate with the awardee for an indirect cost rate audit for an NSF cognizant awardee.
Nevertheless, we are offering comments to help ensure that NSF' s policies and procedures will
provide reasonable assurance that the controls in place, both during construction and operations,
will strengthen NSF's and its awardees' ability to exercise proper stewardship of large facility
funds. Our comments are provided below by section. We hope you find them useful.
Federal Register Notice Vol. 81, No. 89, dated May 9, 2016, Comment Request
The Comment Request Summary states that NSF will be "requesting OMB clearance of this
collection for no longer than 3 years." We suggest that NSF also state its intentions after that time
period (in a later version of the reporting tool), especially since the subsequent charge statement
is more focused on determining and assessing awardee burden, rather than on emphasizing NSF's
need to improve its, and the awardee's, financial management of large facility awards. The
instructions should explain that the reporting tool was instituted to ensure that funds for large

facility awards. The instructions should explain that the reporting tool was instituted to ensure
that funds for large facilities are properly expended and managed. This message was verbally
conveyed during NSF's recent Large Facility Workshop, but was not communicated as well in
the written request for comments from the community.
Instructions and Instruction Overview:

Overall, the reporting tool collects budgeted and actual costs for individual Cooperative
Agreements (CA)/Cooperative Support Agreements (CSA). However, NSF does not clearly
explain that the reporting tool also provides for budget to actual comparison/reconciliation at the
total project level, which is the level cumulative proposed budgets are made and National
Science Board approval is obtained. Therefore, the instructions should be clarified to explain
that NSF intends to fully account for all of the awards made for each NSF large facility that are
in total valued at $100 million or greater and may be comprised of several different CSA and CA
awards, funded by multiple NSF appropriations.
Additionally, the reporting tool does not collect "Contingencies" or "Fee" data (either in budget
or in expenditures to date). The tool also does not collect drawdown data or cash on hand data
that could be reconciling items when comparing budget and actual costs and costs claimed to the
general ledger. Without this information it will not be possible to adequately reconcile and
compare actual to budgeted costs to determine how funds are being spent in comparison to how
they were planned.
In the Overview Section, specific criteria for applicability of the reporting tool should be
clarified, i.e. the tool is required for each CSA and CA for a facility or project that has a total
awarded amount of $100 million or more. It should explain that NSF will identify all the awards
that comprise each project and modify the award agreements to identify each CSA and CA that
require completion of the reporting tool. This section should state that one other purpose of the
tool is to provide for a comparison of expenditures to budget amounts for each award and the
total of the project's awards, to the maximum extent practicable.
Also, the Instruction Overview states, "Only this NSF-approved tool should be used to submit
incurred cost data." This statement should be clarified because it could create confusion if a full
incurred cost submission is required from an awardee for which NSF is cognizant for the
purposes of negotiating the awardee' s indirect cost rate.
CSA Budget Worksheet, CSA Cost Worksheet, and Program Expenditures
Worksheet: Under major costs elements, the tooldoes not provide lines for "Contingencies" or
"Fee" (either in budget or in expenditures to date), although these may be significant items
included in some CAs. Contingencies are required budget information for construction
awards specifically listed on Form SF 424C (line 13), which NSF has agreed to use and should
be collected by this tool. Fee should also be included in the tool if a recipient proposes fee, since
this is also a cost to NSF. Additionally, we note that the tool does not provide for cost
drawdowns (ACM$), which should be collected for comparison to costs claimed.

2

The instructions for worksheet preparation for the CSA Cost Worksheet state that the Program
Charges Worksheet consolidates this data for all CSAs. There is no Program Charges
Worksheet, so the instructions should be clarified (perhaps the Program Expenditures
Worksheet).
It appears, based on the color coding legend provided, that the current expenditure information in
the CSA Budget Worksheet is linked from other report tabs and the current claimed amount on
the CSA Cost Worksheet is auto-calculated. The tool should explain this and caution the
awardee to ensure that the calculated expenditure amount agrees with the amounts claimed. Also,
the reporting tool should provide for an explanation in the event there is a difference between the
General Ledger and Job Cost Ledger on the CSA Cost Worksheet.

It would also be useful if the tool used consistent terms throughout the different worksheets to
identify current costs claimed/expenditures and to ensure that the title of the worksheet tab
matches the title on the worksheet.
Subcontracts and Consultants Worksheet: We note that worksheet does not require a specific
classification of subcontracts vs. consultants. This distinction is important and needs to be made
for indirect rate application purposes because modified total direct costs (MTDC) include the
first $25,000 of sub grants/Subcontracts, while the remaining portion of Subgrants/Subcontracts
over $25,000 is excluded. Additionally, the worksheet does not request the type of subcontract
(CPFF, firm fixed-price, etc.). It appears the worksheet only provides for cost reimbursement
subcontracts, but there is no place for profit which should be collected because it is part of total
cost. There should also be a place on the worksheet for firm fixed-price subcontracts. Finally,
the worksheet should also include a checkbox to document whether the Uniform Guidance
required cost/price analyses in excess of the Simplified Acquisition Threshold (currently
$150,000) are maintained in the file and are available upon request.
Subawards Worksheet: This worksheet should also include a checkbox to document whether
the Uniform Guidance required cost/price analyses in excess of the Simplified Acquisition
Threshold (currently $150,000) are maintained in the file and are available upon request.
Indirect Cost Rate Worksheet: We note that worksheet provides "rate applied" and "total
expended," but does not request the type of rate applied (e.g., provisional, predetermined fixed,
carry forward, etc.), which should be requested. Also, the worksheet only addresses three types
of indirect rates (classified as F&A, G&A and ID other). Some organizations may have
additional indirect rates that should be disclosed, so the tool should be modified accordingly.
Certification of Costs: Suggest expanding certification to include award terms and conditions,
NSF policies and recipient policies.
If you have any questions or wish to discuss our comments, please contact Jannifer Jenkins, at
703-292-4996.

3

National Science Foundation • Office oflnspector General
4201 Wilson Boulevard, Suite 1-1135, Arlington, Virginia 22230

MEMORANDUM

DATE:

July 8, 2016 ·

TO:

Ms. Suzanne H. Plimpton, Reports Clearance Officer

MW_@.
FROM:
SUBJECT:

tant Inspector General for Audit (Acting)
Office oflnspector General Comments on the Draft Large Facility Manual

On behalf of the National Science Foundation Office of Inspector General (NSF OIG), I am
submitting the attached comments on the draft Large Facility Manual (LFM) dated May 2016.
Overall, we appreciate the increased oversight incorporated into the LFM. However, we note that
the LFM is unclear as to which of the new requirements relate to large facilities in operations and
suggest NSF clarify this prior to the LFM issuance. We also suggest ensuring that LFM guidance
for NSF staff and recipients aligns with NSF internal Standard Operating Guidance (SOGs). As
an example, see our comment on section 4.2.4.1 , NSF Policy Positions. We hope you find our
comments and suggestions useful.

Attachment

Section/Page Number

Language in Manual/Observations

Comments and Suggestions

Summary Timeline for
MREFC Projects, 2.1.4-2

NIA

We suggest updating figure 2.1.4-1 to incorporate
new requirements, such as the independent cost
estimate, incurred cost audits, incurred cost
submissions, etc.

4.2.1 Cost Estimating and
Analysis Overview, 4.2.12

When submitting packages for cost analysis,
Recipients must submit the following as a
mm1mum:
• Cost Estimating Plan per Section 4.2.2.1.
• "Cost Model Data Set" per Section 4.2.2.1.
• Reports and Proposals per Sections 4.2.2.2
and either 4.2.3.2 or 4.2.4.2.
• The Work Breakdown Structure (WBS) and
WBS Dictionary per either Sections 4.2.3.3
or 4.2.4.3.
• Supporting information forming the Basis of
Estimate (BOE) per Sections 4.2.2.4,
4.2.2.5,4.2.2.6, and either 4.2.3.4 or 4.2.4.4

Please clarify whether the steps apply to facilities in
construction, operations, or both. For example, it
was unclear in the operations section whether WBS
is required for operations awards or not.

4.2.2 Cost Estimating and
Analysis Process, 4.2.2-10

As a term and condition of the award, the
recipient will be required to provide information
(typically annually) on the actual use(s) of the
management fee.

We suggest NSF more definitively outline the
frequency of submitting management fee data. We
suggest annually instead of "typically annually".

The LFM does not require the auditee to submit
a written assertion of need that details all sources
of revenue, and examine all federal and nonfederal sources of revenue for each awardee in
determining whether a management fee is
necessary and warranted.

We suggest NSF require the auditee's proposal
involve a full disclosure of all financial resources
and other sources of income available to cover
unallowable expenses.

Page 1

The LFM lists appropriate uses of management
fee stating, "Examples of potential appropriate
needs include .... and financial incentives to
obtain and retain high caliber staff."

4.2.2.-7 Cost Estimating
and Analysis Process,
G.6-0ther

4.2.2.1 Cost Estimating
Plan, 4.2.2-1 and 4.2 .. 3-2

Page 2

Our review of management fee uses at large facility
awardees highlighted examples of "incentives,"
including significant signing bonuses and payment
for private school tuition of employees' children.
The use of management fees for these types of
expenses may be questionable to taxpayers. We
suggest NSF consider giving examples of
"appropriate" financial incentives.
The LFM does not address whether management We suggest NSF require management fees to be
fee should be captured under a separate line item submitted as a separate line item in its future NSF
proposals, rather than including them under a
in proposals.
category of allowable cost.
The contingency estimates include indirect costs as
Budget contingency should also be presented as
well as direct costs, so it is not correct to categorize
part of the total amount of Other Direct Costs
and included in section G.6 on the standard NSF contingency as only "Other Direct Costs."
Contingencies are required budget information for
budget form.
construction awards specifically listed on Form SF
424C (line 13) which NSF has agreed to use. We
suggest that budget contingency have its own
separate line item on proposed budgets to NSF.
Please
clarify whether the GAO Cost Estimating
• For construction awards, the CEP must
and Assessment Guide is a requirement for
explain how the Recipient will follow "The
facilities in construction, operations, or both. On
Twelve Steps of a High-Quality Cost
pg. 3 .5 .1-1 in the operations plan section it states
Estimating Process" from the GAO Cost
"Note that cost estimating should follow the
Estimating and Assessment Guide: Best
Government
Accountability Office (GAO) Cost
Practices for Developing and Managing
Capital Program Costs (GA0-09-3SP March Estimating Guidelines, per Section 4.2." However,
4.2 is unclear what applies to construction and what
2009, or subsequent revision).
applies to operations. In addition, while the
• Recipients are required to follow the best
construction section requires a statement from the
practices within the GAO Cost Estimating
awardee that they followed the GAO Cost
and Assessment Guide: Best Practices for
Developing and Managing Capital Program

4.2.2.2 Relevant
Guidance and Reporting
Formats

Cost Estimating and
Analysis for Operations
Awards, 4.2.4-1 through
4.2.4-4.

Page 3

Costs (GA0-09-3SP March 2009, or
subsequent revision) and GAO Schedule
Assessment Guide: Best Practices for Project
Schedules (GA0-16-89G December 2015, or
subsequent revision), taking into
consideration NSF policy and practice as
provided in this manual.
• In the cost estimate section on construction it
states, "The additional high level information
should typically be provided .... A statement
positively affirming recipient compliance
with applicable federal regulations and NSF
policy, including the LFM and GAO Cost
Estimating and Assessment and GAO
Schedule Assessment Guides." This is not
required in the operation's estimates section
of the LFM.
Budget contingency should also be presented as a
part of the total amount of Other Direct Costs and
included in section G.6 on the standard NSF budget
form. Budget contingency budget estimates should
be developed in accordance with Sections 4.2.5 and
5.2 of this manual. Budget contingency and
allocations of contingency will be called out in the
Cooperative Support Agreement by the GIAO under
the "Contingency" section, based on information
provided in the negotiated budget justification.
The language in the construction estimate
section includes:
• "The results of an independent cost estimate
review will be factored into the NSF cost
analysis. To ensure maximum usefulness of

Estimating and Assessment Guide the same
requirement is not made for operations.

Comment: The new Large Facility Manual outlines
how NSF awardees should develop contingency in
Sections 4.2.5 and 5.2 including development of a
risk management plan, tying contingency to WBS
elements, use of Monte Carlo simulations etc. We
suggest 4.2.2.2 be clear that the proposal budget
should include adequate documentation to support
that the contingency amounts were developed in
accordance with 4.2.5 and 5.2 and are supportable.
The cost estimating and analysis for operations
awards section is silent on whether independent
cost estimate reviews will be done on operations
awards. We suggest NSF require that significant
operation awards go through an independent cost
estimate review.

•

the review, it will generally be done as part
of the Preliminary Design Phase."
"There may be one or more years between
the PDR and the start of construction, which
is predicated on successful completion of the
FDR."

4.2.4.1 NSF Policy
Positions, Point 4

In support ofNSF's "No Cost Overrun" policy,
projects shall use a confidence level for
contingency estimates between 70 and 90
percent (under a probabilistic approach).

4.2.4.1 NSF Policy
Positions, Point 5

NSF will hold budget contingency through
project completion, in an amount up to 100% of
the total NSF-approved contingency budget,
until it can be justified for obligation.
However, the LFM does not outline the
procedures related to this new policy; for

Page 4

We suggest NSF should make it clear when the
independent cost estimate will be done so awardees
can plan accordingly. Currently, the LFM says
"will generally be done as part of the Preliminary
Design Phase."
If there are significant changes between the PDR
and FDR how will NSF assess the new quotes and
data? We suggest NSF conduct an independent
review of all updated quotes.
If the independent cost estimate review indicates
significant issues, or significant time has elapsed
between the PDR and FDR, how will NSF handle
this? We suggest NSF request the recipient to
update its estimates with current, accurate and
complete cost data and also perform a follow-up
review.
We suggest that the large facility proposals go
through the most stringent of the eight types of
GAO independent cost reviews, an independent
cost estimate.
We suggest NSF also state its expectation not only
in terms of a confidence level, but also the
precision of the estimate (so that NSF can say, for
example, that it is 80% confident that the point
estimate is accurate within a range of plus or minus
20%).
We suggest describing the process and procedures
related to this new policy of holding budget
contingency, including explaining how the amount
of contingency to be held by NSF will be
determined. Also, indicate that the allocations
would be included in the award notice and the CSA

example, that the allocations would be included
in the award notice and the CSA amended. As
another example, the new SOG states the project
would monitor budget contingency against both
the NSF-approved contingency budget and the
total allocation to date. This expectation should
be made clear in the LFM.

4.2.5.4 Contingency
Planning and Assessment
during Preliminary
Design

4.2.5.5 Development of
the Contingency Use
Process

4.2.5.6 Contingency
Planning and Assessment
during Final Design

Page 5

amended. Additionally, the new Contingency SOG
(under Monitoring and Reporting of Budget
Contingency) requires that the Project monitor
budget contingency use against both the NSFapproved contingency budget and the total
allocation to date. This expectation should be made
clear in the LFM. We suggest the LFM be updated
to align with the SOG, i.e., incorporate
requirements from the new SOG as it relates to the
recipient.

We suggest NSF make it clear whether
Use ofrigorous probabilistic cost estimating
methods that estimate confidence levels for the
probabilistic cost estimating methods are, or are
TPC (such as Monte Carlo methods based on
not, a requirement. If probabilistic cost estimating
is not used, what are other acceptable methods?
probability distributions for risk) are preferred
and NSF highly encourages application of these
methods where practical.
When use of budget contingency is requested, we
The CCB change request document, whether
forwarded to NSF for approval or not, must have suggest that NSF require the recipient to document
in the Change Control Request (in Additional
the minimum content requirements necessary to
Documentation for example) which of its accounts
comply with relevant cost principles, as well as
the related charges will be recorded under to
to maintain an audit trail. SEE SAMPLE
CHANGE CONTROL REQUEST FORM. This provide better traceability in its accounting system.
form states "Cost control accounts may be
included for traceability in the accounting
system."
It would be useful if the contingency section
At the Final Design Review (FDR) the budget
explained whether the independent cost estimate at
estimate should be substantially based on
PDR would cover a review of contingency. Also, if
externally obtained cost estimates (vendor
contingency amounts are further refined between
quotes, bids, historical data, etc.). This added
PDR and FDR, we suggest stating that NSF will
definition is expected to result in an increase in
review the revised amounts and specifying what
the project's estimated Budget at Completion
(BAC) and a reduction in its budget contingency, type of review will be performed.

4.2.5.7 Contingency Use
and NSF Oversight
during Construction
4.2.5.8 Reporting
Requirements

4.4 DOCUMENTATION
REQUIREMENTS

4.5.3 Recipient
Performance Reviews
and Audits, 4.5.3.4

4.5.3 Recipient
Performance Reviews
and Audits, 4.5.3.5

Page 6

while TPC remains constant. Also as part of the
FDR, NSF assesses the methodology employed
by the project to further refine its cost and
contingency estimates including schedule and
scope adjustments.
Unexpended contingency funds may not be used
to support operations or other out-of-scope
activities.
At a minimum, the monthly report will include:
... (4) an updated change log indicating all
contingency allocations ("puts and takes") and a
"liens" list of projected amounts of possible
future calls on contingency.
In addition, access to any pertinent books,
documents, papers and records should be made
available to the NSF Director and the
Comptroller General of the United States or any
of their duly authorized representatives to make
audits, examinations, excerpts and transcripts in
accordance with either the Uniform Guidance or
FAR requirements.
NSF conducts a cost incurred audit for large
facility awards above $1 OOM at the end of the
award and potentially during execution of the
award based on an annual large facility risk
assessment conducted by the Large Facilities
Office and the Cooperative Support Branch at
NSF.
EVM section stated "This section reserved for
future content.

We suggest that NSF clearly define scope and outof-scope activities to avoid the possibility of
subsequent disagreement between NSF and the
recipients as to the meaning of these terms.
We suggest that NSF also request its recipients to
provide a comparison of approved budget to actual
use of contingency. This comparison could be the
recipients' affirmation that they are using
contingency funds as approved by NSF.
We suggest adding NSF OIG to the list of
organizations for which the awardee must provide
access to documentation.

Please clarify whether the cost submission and cost
incurred audits apply to facilities in construction,
operations, or both. We suggest both.

As we have recommended in the past, we suggest
the certification of EVM systems and the validation
ofEVM data.

4.5.3.2 NSF External
Reviews.

Additional ad hoc reviews may be requested by
the PO under certain circumstances, such as
significant re-planning of construction projects,
changes in key personnel, and major changes in
research technical design, direction and scope.

4.5.5 Re-Baselining

The re-baselining section does not explain what
type of reviews NSF will complete in the event
of a re-baseline. For example, whether rebaselined proposals will be required to go
through an independent cost review.
For MREFC construction projects, the amount of
budget contingency is determined by performing
a probabilistic risk analysis on the baseline cost
and schedule and selecting a Total Project Cost
with an acceptable confidence level (typically
between 70-90%).

5.2.3.2 Contingency
Definitions

5.2.6.3 Risk Description

However, note that NSF does not allow the use
of contingency for risks that are commonly
referred to as "unknown unknowns" such as
exceptional events or major changes in scope.

5.2.8.8 Handling Inflation

Inflation is part of the NSF budgeting and
project planning.

5.2.11.1 Contingency
Budget Timeline

At the Final Design Review (FDR) the PEP
budget estimate should be substantially based on

Page 7

We suggest review of updated vendor quotes/bids
be added to the circumstances. As also referred to
in Section 4.2.5.7, we suggest the NSF clearly
define scope, change in scope and major change in
scope, to avoid the possibility of subsequent
disagreement between NSF and its recipients as to
the meaning of these terms.
We suggest clarifying what types of reviews rebaselined proposals will go through.

As also mentioned in 4.2.5 .1, we suggest NSF also
state its expectation not only in terms of a
confidence level, but also the precision of the
estimate (so that NSF can say, for example, that it
is 80% confident that the point estimate is accurate
within a range of plus or minus 20%).
Suggest that NSF provide some examples of
exceptional events for which contingency is not
allowed to be used. Also, as mentioned in 4.2.5.7
and in 4.5.3.2, we suggest the NSF clearly define
scope, change in scope and major change in scope,
to avoid the possibility of subsequent disagreement
between NSF and its recipients as to the meaning of
these terms.
We suggest that NSF require its recipients to affirm
(and also verify in NSF reviews of proposal
budgets) that inflation is not double-counted in
other estimates, such as in contingency.
Consistent with our comments in 4.2.4, we suggest
that NSF instruct its recipients that estimates based

externally obtained cost estimates (vendor
quotes, bids, historical data, etc.)

Page 8

on vendor quotes/bids should be kept current (i.e.,
preferably less than a year old but no more than 18
months) throughout the project life cycle at least
until award.

From:
To:
Subject:
Date:

Hawkins, Matthew J.
Wilkinson, Carol; Yasky, Rebecca Kay
FW: Comments on the Large Facilities Manual draft and more
Wednesday, August 31, 2016 8:43:19 AM

Carol and Rebecca,

I believe I sent this earlier for Carol’s consideration & incorporation  (I think valuable to include
OISE’s engagement), but re-sending just in case.

Matt

From: Sharma, Mangala
Sent: Tuesday, August 30, 2016 4:36 PM
To: Hawkins, Matthew J.
Subject: Comments on the Large Facilities Manual draft and more

Hi Matt,
I’m not sure if you got OISE comments (approved by Rebecca Keiser) on the draft
LFM. So, I’m (re-)sending this mail. If, on the other hand, you already did receive
these, apologies for the duplication!

Please don’t hesitate to let me know if you have any questions or need clarification
about our comments. If you have any concerns about the recommendation that
OISE be included in the IPT and LFWG, I’d be open to suggestion on a more informal
role for OISE as well.

Thanks very much!
Mangala

——
Thank you for the opportunity to comment on the 2016 draft of the manual.
Overall, the NSF Office of International Science and Engineering (OISE) appreciates
that the LFM makes thoughtful recommendations about international partnerships
and engagements. OISE appreciates, especially, inclusion of international
considerations in:
Sec. 2.1.6.4 and Appendix A – MREFC panel roles and criteria for prioritizing MREFC
projects;
Sec. 2.3 – Design Stage;

Sec. 4.6 – Partnerships;
Sec. 5.5 – Environmental Considerations; etc.

We do request that the role of OISE in facilitating and coordinating NSF international
research infrastructure be explicitly included both (i) in practice i.e., how LF projects
are overseen and (ii) in the LF manual – which are of course related. For (ii), we
recognize that, for the most part, specific divisions/offices within NSF are not called
out in the manual.

Regarding specific LF projects, could we please consider adding an OISE program
officer to each LF Integrated Project Team (IPT)? This OISE PO could provide
country-specific guidance on international partnership for a particular MREFC
project, and as necessary, facilitate coordination with international partners during
the design-to-execution phases of the project. If such a role for OISE is agreeable to
LFO, then, a suggested edit to the LFM is:
Sec.2.1.6.2 - Integrated Project Team (IPT) - “Comprised of NSF personnel with
knowledge and expertise in areas related to the scientific and technical, award
management, international partnership, and strategic aspects of a particular
MREFC project.”

We also request nominating to the overall Large Facilities Working Group the
OISE program officer who is assigned to be OISE liaison to LFO. This is distinct
from the OISE participation on specific IPTs (above). As part of the LFWG advisory
group, the OISE PO could provide big-picture input regarding international
cooperation, US foreign policy, etc., related to NSF Large Facilities.

Other editorial comments on the draft manual:

Pg. 157 – Please correct the name of the office; “NSF Office of International and
Integrative Activities (OIIA)” should be “NSF Office of International Science and
Engineering.”

Pg. 22 - end of para 1 - "(ideally) were partnerships" should be "(ideally) where
partnerships"

From:
To:
Subject:
Date:

Porter, Kevin
Wilkinson, Carol
LFM PEP Guidance & Draft Comments?
Monday, August 08, 2016 3:47:37 PM

Hello Carol,

I had some LFM related questions for you…

Section 3.4.2 refers to an internal NSF guidance document “Guidelines for Development of Project
Execution Plans for Large Facilities.”  Can you provide me a copy of this?  Is this even useful and still
worth sharing with Programs internally at NSF?  Should this reference be deleted from the LFM and
become a SOG?

I haven’t seen any questions/comments on my draft Section 4.2.  Do you have any that I should be
working on?

I have some other edits and minor additions to 4.2 (e.g., tweaking some definitions, clarify
applicability to contracts, referencing AACEI guidance in a few areas, adding expectations for
estimate method typically used at each stage) – is there some protocol I need to follow to
incorporate these and address the other public comments?

Thanks,
Kevin

Large Facilities Manual: NSF 16-0XX (May 2016)
2.2.2 Exit from Development to Design Stage
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
{BFA-LFO)

2.2.2

Exit from Development to Design Stage

Formal start of the Design Stage for a facility project occurs once a recommendation of the
MREFC Panel and approval by the NSF Director is received . This process is initiated by a request
from the sponsoring Directorate and/or Division to the Director's Office once a project is
determined to be ready for the Conceptual Design Phase and potential construction with
MREFC funds. Generally, such a request is made when the sponsoring organization has
determined that: (1) the project is a high priority for further development, (2) the project is
eligible for MREFC funding (see criteria) and the MREFC funding route is preferred, and (3) the
sponsoring organization is committed to begin explicit investment in more detailed design
activities in the current or upcoming budget cycle using Directorate or Divisional funding
(R&RA) .
The MREFC Panel's recommendation will focus on providing the Director with answers to the
following questions:
Science
•
•

Is there a compelling science case, and are the project' s goals well-articulated?
Does the project fit solidly within the NSF "mission," within the strategic plans of the
NSF and that of the sponsoring Directorate or Division, and within th e broader NSF
facility portfolio?

Planning
•
•

•
•

Is the sponsor's plan for stewardship of the Conceptual Design Phase consistent with the
guidelines set out in the Large Facilities Manual?
Does the preliminary timeline for development and implementation include
programmatic, NSB, budget and any necessary partnering milestones, including explicit
project off ramps?
Are potential opportunities for internal and or external partnering being considered, if
not already underway?
Are there any conflicts of interest or other major challenges regarding this project that
t he Director needs to be aware of?

Based on the Panel's recommendation and any further examination, the Director then approves
(or disapproves) the project entering the Conceptual Design Phase as a "candidate" MREFC
project . Note that no NSF commitment is implied beyond support for the development of a
Conceptual Design. The MREFC Panel or Director might alternatively advise the sponsoring
organization to look further into an issue or issues and then return to the Panel for further
consideration.

Section Revision:
June 17, 2015

2.2.2-1

From:
To:
Subject:
Date:
Attachments:

Hawkins, Matthew J.
Wilkinson, Carol; Plimpton, Suzanne H.
FW: "For comment" draft of the Large Facilities Manual (LFM) published in the Federal Register
Thursday, May 19, 2016 2:57:34 PM
image001.png

For the record.

Matt

From: Sholhead, John E
Sent: Thursday, May 19, 2016 1:45 PM
To: Hawkins, Matthew J.
Cc: Howe, Michael ; Carpenter, Kathleen Elizabeth
Subject: RE: "For comment" draft of the Large Facilities Manual (LFM) published in the Federal
Register

Hi Matt:

Mike H has three comments (below) for the Large Facilities Manual (LFM). Other than these, we
think it looks good.

Thanks,
John

4.2.2.4 Supplementary Guidance for Construction and Operations Awards:  Recommend adding
specific PAPPG and/or Uniform Guidance references to the sub paragraphs.  For example 4.2.2.4.ETravel:  could reference the PAPPG section on Travel (Chapter II.C.2.g.(iv)) and/or perhaps 2 CFR §
200.474.

4.5.3 Recipient Performance Reviews and Audits:  Recommend considering adding Reviews for
Payment Testing and for Program income Reporting.  This will increase awareness of these
monitoring processes and establish expectations for recipient responses to the reviews.  We can
provide language should this be recommendation be accepted.  

A general observation:   Risk assessment and monitoring,  as well as contingency, are addressed in
various sections of the manual.  While project lifecycle has a major impact on the subjects, it may be
more clear to consolidate risk and contingency into a single subject area in order to provide focus.    

From: Hawkins, Matthew J.
Sent: Monday, May 16, 2016 10:21 AM
To: NSF SMART ; NSF Deputy AD ; NSF Division Directors
; NSF Program Officers ; NSF AMG ; NSF AO/OS
; NSF Deputy Division Directors ; BFA ALL ;
OGC All ; DIS ALL ; DAS Staff ; BFA BPLG

Cc: NSF POLICY ; Hawkins, Matthew J.
Subject: "For comment" draft of the Large Facilities Manual (LFM) published in the Federal Register

Dear Colleagues:

On May 9th, NSF published a notice in the Federal Register announcing the availability of a “For
comment” draft of the Large Facilities Manual (LFM).   We are soliciting Directorate/Office
comments on the draft LFM until COB July 8, 2016.   Please note that all comments should be
coordinated through the applicable Assistant Director/Office Head’s office and sent to  Suzanne
Plimpton at [email protected].

The following are links to the Federal Register Notice and the draft LFM:

· https://www.gpo.gov/fdsys/pkg/FR-2016-05-09/pdf/2016-10793.pdf
· http://www.nsf.gov/bfa/lfo/lfo_documents.jsp

Any technical questions should be directed to Dr. Carol Wilkinson in the Large Facilities Office at
[email protected].

Kind Regards,
Matt Hawkins

nsflogo

Matthew Hawkins
Head, Large Facilities Office
Office of Budget, Finance & Award Management
voice: x7407
email:  [email protected]

From:
To:
Cc:
Subject:
Date:

Whitehurst, Eddie
Hawkins, Matthew J.
Wilkinson, Carol; Rabanal, Florence I.; Kinser, William; Lupis, Jeffery Michael
RE: LFM
Monday, May 16, 2016 9:10:40 AM

Matt
Thanks glad to know we can make these small changes.  Sometimes the word is used in relation to
the award or project and closeout is the official word which should be used to denote the end of the
award.  These sections should be using the word “closeout” the red words should be deleted and the
blue added.

2.1.6-5
Table 2.1.6-1

Devises and carries out a renewal or termination strategy that implements recompetition of the
operating award wherever feasible

2.1.6-7
Devises and carries out a renewal or termination strategy that implements recompetition of the
operating award wherever feasible
Defines business practices for renewal, recompetition, or termination of the Award
Advises PO and G/AO on effective operational oversight strategies, renewal and recompetition
strategies, and termination

2.1.6-10
Overseeing implementation, operation, and eventual divestment and termination of NSF support for
the project.

2.1.6-13
The G/AO confers with the PO and other relevant offices to ensure that the NSF’s technical and
administrative oversight activities are well coordinated. The G/AO and the PO collaborate on the
preparation of solicitations and the proposal and award process. The G/AO has individual
responsibility for developing and overseeing the implementation of financial and administrative
aspects of the award process, and joint responsibility with the PO for recompetition planning and
execution, and award termination and closeout.

2.3.1-3
facility. It also identifies critical issues and risks facing the project (for example: project management
issues, completing essential R&D activities, partnership agreements, termination or divestment
liabilities) and lays out a strategy for financing these activities.

3.1-1
Guidelines for plans to terminate operations under NSF awards are in development, with Section 3.6
provided as a placeholder. Divestment, and termination of NSF funding and oversight of a facility,
may be accomplished through transfer to another agency or funding source or through
decommissioning and deconstruction.

3.3-1
This document provides guidance to the PO on topics to be included in an Internal Management Plan
(IMP), grouped by life-cycle stage. The IMP is the primary document that describes how NSF will
oversee development, construction, operation and eventually divestment and termination of the
award support for a major facility. The requirement to develop an IMP is described in Section 2.3.1
for MREFC and in Section 2.7 for non-MREFC projects. Two primary purposes are served by
development of an IMP:

4.2.2-1
The CEP should be tailored to address all relevant stages and costs of the facility lifecycle, from
Development and Design through Construction, Operation, and Termination Disinvestment. For
example, the expected level of funding needed for the Operations Stage should be identified at the
Conceptual Design Review. Operating costs estimates will be updated throughout the design and
construction process as further discussed in the Concept of Operations Plan developed as part of the
PEP described in Section 3.4 of this manual.

4.2.2-2
For most large facilities projects, funding is derived from the appropriate NSF budget account
depending on the Stage: typically the Research and Related Activities (R&RA), Education and Human
Resources (EHR) or MREFC account. Barring documented exceptions, the R&RA (and possibly EHR)
account will be used to fund the Development, Design, Operations, and Termination Disinvestment
Stage costs. The MREFC account will be used to fund construction, acquisition and commissioning
costs as part of the Construction Stage.

4.2.4-1
If anticipated by NSF and as discussed in Section 2.6 of this manual, proposals will be requested to
address partial or full termination disinvestment of the facility during the award period, including
property decommissioning and disposition costs and other costs related to employee separations. It
is incumbent upon the Recipient to ensure their operations proposal is complete, appropriate, and
reasonable.

4.5.1-1
The Recipient is responsible for complying with the reporting requirements contained in the award
instrument (e.g., technical and financial reporting), this manual, and in the Proposal and Award
Policy and Procedures Guide (PAPPG) – particularly with respect to property management and final
reporting and closeout requirements for termination of the award. The Recipient is also responsible
for providing internal oversight of its own activities.

Eddie Whitehurst
Deputy Branch Chief
Cooperative Support Branch
National Science Foundation
4201 Wilson Boulevard, Room II-300.27
Arlington, VA 22230

(703) 292-2292

From: Hawkins, Matthew J.
Sent: Friday, May 13, 2016 3:30 PM
To: Whitehurst, Eddie
Cc: Wilkinson, Carol ; Rabanal, Florence I. ; Kinser, William
; Lupis, Jeffery Michael
Subject: RE: LFM

Eddie,

The last stage in the MREFC process has been changed to “Divestment”.  Are you looking at the
posted revision?

If so, do you see the term “termination” used elsewhere inappropriately?  If so, please point it out.

We have the opportunity to tweak after the public comment period, so no worries!

Matt

From: Whitehurst, Eddie
Sent: Friday, May 13, 2016 9:38 AM
To: Hawkins, Matthew J.
Cc: Wilkinson, Carol ; Rabanal, Florence I. ; Kinser, William
; Lupis, Jeffery Michael
Subject: LFM

Matt
I know the LFM is out for public comment.  Will we get one last chance for a few edits or changes
based on comments or further clarification.
I have one item which I forgot to include because my notes where in an email I could not find during
the crazy week of adding the Large Facilities Financial Data Tool.

The concern is the use of the word Termination as part of the construction stage.  The new 2 CFR
200 which was approved last year standardized all Federal definitions.  Agencies cannot be at odds
with the use of the guidance.  The LFM use of the word Termination is going against what I think it is
you want to convey.  The last stage of the construction process is the Closeout of the award not a
termination.
The use of the word Termination is a negative and means you end the award before the grantee
could finish the project. See below the definitions.

§200.16 Closeout.
Closeout means the process by which the Federal awarding agency or pass-through entity determines
that all applicable administrative actions and all required work of the Federal award have been completed
and takes actions as described in §200.343 Closeout.

§200.95 Termination.
Termination means the ending of a Federal award, in whole or in part at any time prior to the planned end
of period of performance.

Thanks
Eddie

Eddie Whitehurst
Deputy Branch Chief
Cooperative Support Branch
National Science Foundation
4201 Wilson Boulevard, Room II-300.27
Arlington, VA 22230
(703) 292-2292

From:
To:
Cc:
Subject:
Date:

Hawkins, Matthew J.
Whitehurst, Eddie
Wilkinson, Carol; Rabanal, Florence I.; Kinser, William; Lupis, Jeffery Michael
RE: LFM
Friday, May 13, 2016 3:30:04 PM

Eddie,

The last stage in the MREFC process has been changed to “Divestment”.  Are you looking at the
posted revision?

If so, do you see the term “termination” used elsewhere inappropriately?  If so, please point it out.

We have the opportunity to tweak after the public comment period, so no worries!

Matt

From: Whitehurst, Eddie
Sent: Friday, May 13, 2016 9:38 AM
To: Hawkins, Matthew J.
Cc: Wilkinson, Carol ; Rabanal, Florence I. ; Kinser, William
; Lupis, Jeffery Michael
Subject: LFM

Matt
I know the LFM is out for public comment.  Will we get one last chance for a few edits or changes
based on comments or further clarification.
I have one item which I forgot to include because my notes where in an email I could not find during
the crazy week of adding the Large Facilities Financial Data Tool.

The concern is the use of the word Termination as part of the construction stage.  The new 2 CFR
200 which was approved last year standardized all Federal definitions.  Agencies cannot be at odds
with the use of the guidance.  The LFM use of the word Termination is going against what I think it is
you want to convey.  The last stage of the construction process is the Closeout of the award not a
termination.
The use of the word Termination is a negative and means you end the award before the grantee
could finish the project. See below the definitions.

§200.16 Closeout.
Closeout means the process by which the Federal awarding agency or pass-through entity determines
that all applicable administrative actions and all required work of the Federal award have been completed
and takes actions as described in §200.343 Closeout.
§200.95 Termination.
Termination means the ending of a Federal award, in whole or in part at any time prior to the planned end
of period of performance.

Thanks
Eddie

Eddie Whitehurst
Deputy Branch Chief
Cooperative Support Branch
National Science Foundation
4201 Wilson Boulevard, Room II-300.27
Arlington, VA 22230
(703) 292-2292

Large Facilities Manual: NSF 16-0XX (May 2016)
2.1.3 Facility Life Cycle
Prepared by the Large Facilities Office in the Budget, Finance, and Award Management Office
(BFA-LFO)

2.1.3 Facility Life Cycle
A facility’s lifetime is characterized by the following life cycle stages:
1.
2.
3.
4.
5.

Development1
Design
Construction
Operation
Divestment.

Each life cycle stage entails different actions appropriate to the advancement of the project, the
review and approval needed to obtain NSF funding, and the creation of NSF budgets to support
these activities. Entry and exit from each life cycle stage are clearly defined including required
documents and deliverables. A high-level graphic of the progression through the stages is given
below in Figure 2.1.3-1.
Figure 2.1.3-1

Progressive Steps in the MREFC Life Cycle, Showing Review and Decision Points for Exit and
Entry into Each Stage. The Design Stage is highlighted to indicate that it is further broken down
into phases

Development

Design

Design Phase
Reviews
(Program, MREFC Panel
& DRB, OD & NSB)

Construction

Annual
Construction
Reviews
(Program)

Operations

Divestment

Annual Operations
Reviews

Renewal &
Re-competition Reviews

(Program)

(Program, MREFC Panel & DRB,
OD & NSB)

= Review (Stage gate, annual construction, etc.)
= NSF Decision Point (Program, Director, NSB)

Points at which there may be departure from the MREFC process outlined here should be
identified early in the project development and documented as part of the NSF Internal
Management Plan (IMP) Individuals should discuss any proposed departures with the cognizant
Program Officer.
The Design Stage is further divided into three phases, each with defined entry and exit points,
as shown in Figure 2.1.3-2 below:
1. Conceptual Design
2. Preliminary Design
3. Final Design
1

A project in the Development Stage may be referred to as a “Horizon” or a Conceptual Development project in earlier NSF
documents and references.

Section Revision:
May 2, 2016

2.1.3-1