Lockheed Martin AIMS Team Questions on the Large Facilities Manual

Meeting Date: 8/5/2016

LM AIMS team questions on Draft LFM 5_4_2016
General Statement: A majority of the LFM appears to be geared toward cooperative
agreements and not for our situation.
1. Pg 2.1.3-1. Points at which there may be departure from the MREFC process outlined
here should be identified early in the project development and documented as part of
the NSF Internal Management Plan (IMP) Individuals should discuss any proposed
departures with the cognizant Program Officer.
a. Question: Will there be any documentation that the Operations and Divestment
stage are not part of this project? The document says any departure from these
stages would be identified in the NSF IMP but I wasn’t sure if that information
was reflected.
2. Pg 2.1.3-3. Preliminary Design Phase Definition: Further advances the project baseline
definition and the Project Execution Plan. It produces a bottom-up scope, cost, schedule,
and risk analysis of sufficient maturity to allow determination of the Project Total Cost
and overall duration for a given Fiscal Year start and to establish the MREFC budget
request to congress.
a. Question: It is understood we provide a funding profile and spend plan for PDR.
We also understand we baseline at PDR per the LFM for scope, cost and schedule.
Does the LFO allow for changes in the funding and spend plan profiles at the end
of FDR to allow for any updates we may need to make based on the awarded
subcontract?
3. Pg 2.1.6-5. Table –

a. Who is the PO – Scott or Ben? It was confirmed at our meeting on 8/5/16 that Ben is
the Program Officer in our case. What do they mean by the CO “creates

solicitations for enabling research, workshops, summer studies…. (the first
1

Meeting Date: 8/5/2016

bullet). 8/5/16 Scot confirmed the first bullet was more for cooperative agreements
and not required for us.

b. For those items highlighted yellow:
i. Does the PO have authority to approve significant change to the project
baseline? 8/5/16: Response was in our case the PO does not authority to
approve baseline changes.
ii. Who are the external ad hoc panel members? 8/5/16: Requires further
review.
iii. Need to understand the “periodic reviews” of project progress – monthly,
quarterly, annually so we can determine frequency and requirements for
these reviews? 8/5/16: This has not been determined.
iv. Is the GPRA where the requirement for an annual performance plan
comes from? 8/5/16: Requires further review and verification by NSF.
4. Pg 2.1.6-13. The G/AO is responsible for oversight of the financial and administrative
terms and conditions of the assistance agreement,1 just as the PO is responsible for
scientific and technical oversight. Unlike the PO, he/she holds the warrant to obligate
government funds. The G/AO and the PO jointly share the principal technical and
financial responsibilities for the oversight and assurance of a large facility project. In this
capacity, the G/AO is jointly responsible with the PO for the success of a project.
a. What is the difference between the G/AO and the CO?
5. Pg 2.1.6-14. The CO holds the warrant and is the only individual authorized to obligate or
de-obligate government funds. The CO, through their warrant, has the sole authority to
2

Meeting Date: 8/5/2016

award and administer the construction contract(s) used in support of Large Facility
projects.
a. This statement says the “CO has the sole authority to award and administration
the construction contract (s) used in support of Large Facility projects. This does
not make sense for our situation – can NSF clarify the CO’s role in our case.
6. Pg 2.1.6-14. Cost Analyst. The G/AO or CO requests assistance from a NSF Cost Analyst from
the Cost Analysis and Audit Resolution (CAAR) Branch of the Division of Institution and
Award Support (DIAS), located within BFA (CAAR) when cumulative or individual awards
exceed certain thresholds or for Recipients with previously identified risks. The PO, G/AO or
CO, and Cost Analyst all review proposed budgets to help determine if they are allowable,
allocable, reasonable, and realistic for the scope of work. However, the primary purpose of
the NSF Cost Analyst’s budgetary review is to support the G/AO or CO to ensure that the
Recipient has properly estimated and calculated costs and that they are supported and
documented with sufficient rigor. The Cost Analyst provides a written recommendation to
the G/AO stating whether costs are supported or unsupported. The recommendation may
include advice on award terms and conditions or limitations or other concerns identified.
The Cost Analyst may also help determine if the Recipient has adequate business and
accounting systems in place, assess a Recipient’s financial capability and viability, validate
indirect cost rates, or assist in other areas of concern as identified by the requesting G/AO.
a. Who is this on NSF side? 8/5/16. Need to confirm with Kevin that this is not a
cost item that NSF must bear and verify if this person is in place right now.
7. Pg 2.1.6-14 (Footnote).
1 Refer

to the Business Systems Review (BSR) Guide described in Section 4.5.3.3 for discussion on
this point. When NSF is not the cognizant audit agency for the recipient institution, its oversight
of recipient business practices is narrowly defined.

a. Will there be another BSR on ASC as there was already one
conducted when the current contract was awarded? We need to
understand timing as well for when this will take place.
8.

Page 2.3.3-1 4th bullet and 2.3.3-2 footnote comment: A fully implemented PMCS, including
a final version of the resource-loaded schedule and mechanisms for the project to generate
reports – using the Earned Value Management System (EVMS)1 – on a monthly basis and
use them as a management tool. Path dependencies, schedule float, and critical path are
defined; and footnote 1 During construction, progress should be tracked and measured using the Earned Value
method (this method is required by OMB in Planning, Budgeting, Acquisition, and Management of Capital Assets, OMB
Circular No. A–11 (2014). A discussion of Earned Value is included section of Section 8, Guidelines for Earned Value
Management.

a. Will this be a requirement for LM which we will then flow down to our
subcontractors? This is a requirement for the prime contractor (or the award
3

Meeting Date: 8/5/2016

recipient for cooperative agreements). Subcontractors must be able to provide
the appropriate cost and schedule inputs to put into the prime’s EVMS. So this
would apply to LM, but not necessarily subcontractors. (Please note, this
requirement is not new to the 2016 draft, it has been in the LFM in similar form
since at least 2009. Additional guidance for how projects are expected to meet
this requirement is the part currently being developed.) 8/5/16 - Need
definition between certification and validation from the power point
presentation we were shown and which is required and by whom (government
audit agency or third party). Also need to understand how the FAR 34.2 and
DFARS 234.2 words compliant and non-compliant differ from certification and
validation. NSF guidance is being developed to require validation of EVMS to
the intent of the EIA-748 standard. This guidance will explain how the EVMS is
validated for NSF MREFC projects. NSF will not require that EMVS be certified
by DCMA. An expert third party will validate the EVMS meets the intent of EIA748. The EVMS section of the LFM will be comprehensively updated during the
2017 review cycle. However, in the interim, NSF standard operating guidance
will be provided to projects. LFO will also consider adding some clarifying
words to this draft of this 2016 LFM to (1) require validation of the project’s
EVMS to the intent of the EIA-748 standard and (2) allow projects to request
minor deviations with supporting justification. What are the ongoing monthly
ASC (and subcontractor flow-downs) EVMS
requirements/deliverables/expectations? (ie. Will the government auditing
agency or 3rd party agency continue to audit the project monthly, meaning we
deliver all month end business files (including subcontractor?), respond to
findings, work corrective actions, etc. together with NSF EVMS person and
EVMS auditing agency or will the EVMS system be “validated” 1x for the project
and monthly EVMS reporting will continue as per our current
rhythm/deliverables on the ASC contract? The LFM requires LM to provide
monthly EVM reports for construction. Subcontractors must be able to provide
the appropriate cost and schedule inputs to LM for incorporation in LM’s EVMS.
An initial validation will be done in conjunction with FDR to allow resolution of
any findings prior to the start of construction. The validation will include a
“desk review” of plans and procedures followed by a visit to the project to
interview project controls personnel and review the system. Future periodic
surveillance by a third party is expected to be done annually, unless findings
require more immediate or routine follow-up until resolution. If the agency/3rd
party EV company is on contract for the duration of the project, who is the
EVMS NSF contact that ASC would deliver monthly deliverables to and ensure
compliance/work off findings? Expert third party will be contracted by NSF. No
expectation for monthly deliverables except the EVM report per the LFM. LM
should have the appropriate PMO to maintain the PMCS, including EVMS, and
4

Meeting Date: 8/5/2016

issue monthly reports. When does the EVMS system need to be “validated”? Is
it a constraint to exiting PDR/baselining, or required prior to FDR (we will
already be baselined)? In conjunction with FDR, to allow resolution of any
findings prior to the start of construction. Having the EVMS up earlier is
prudent to ensure it is working properly. If EVMS findings are required post
initial baseline, will we have an opportunity to clean up and re-baseline? If
findings are significant and meet criteria in LFM section 4.5.5, yes possibly. But
it is not obvious why a future surveillance of the EVMS might prompt a rebaselining. Note: It is the ASC recommendation that EVM
compliance/validation not be a mandatory subcontract flowdown. Rather ASC
will define requirements for subcontractor submittals and enter the data into
the ASC EVM complaint system. Agree, this has been done successfully for
other projects. This requires clear requirements in subcontracts (e.g., for
timely, discrete cost and schedule inputs that can be physically verified). A
RESPONSE TO THIS IS CRITICAL TO ENSURE WE CAN MEET PDR SCHEDULE AND
INCORPORATE ANY ADDITIONAL COSTS REQUIRED DURING CONSTRUCTION
DEPENDING ON EVMS DETERMINATION. Ben emailed Kevin to set up a
meeting for this specific topic.
b. The bullet would indicate that we are to provide a “final version of the resource
loaded schedule” during FDR. Does this mean we are allowed to update the
“project baseline that is set following PDR (pg 4.2.5-3)? We understand our Total
Project Cost would not change but are we allowed to re-baseline our entire IMS
schedule once we receive a refined schedule from our awarded subcontractor? Is
the intent that the Project (from total project cost and high level scope/schedule
– buildings in/buildings out) is “baselined/locked down” at PDR, but the actual
IMS (detailed tasks, resources, cost spreads/allocations) is not baselined? If so, is
that PDR baseline to remain intact per the “NSF No Cost Overrun” direction for
the entirety of the project? If, at FDR, we learn the General Contractor
bids/schedules are significantly different (more than 10% cost or schedule) than
what we planned and baselined at PDR, are we able to re-baseline to the GC plan,
or are we held to the plan we established at PDR? If/when re-baselining is
necessary, what is the process/timeline to work with the NSB to re-baseline
(approvals, any change boards mtgs/formal change presentations, paperwork) in
order to not compromise delay in funding or in construction on the ice? If we
have to exercise risk contingency, can NSF approve or does the NSB also need to
approve? Can the project continue working when a risk event is realized
exercising risk contingency or is the program on a stop work pending NSF/NSB
approval/exercise of risk item?
c. Are we allowed to carry an amount of “schedule contingency” (ie. time in the
form of an actual “schedule contingency” task) in the IMS? Are we allowed to
carry those schedule contingency tasks throughout the IMS (completion of each
5

Meeting Date: 8/5/2016

building) or a total amount attached to the end of the project? Which date do we
report as our estimated complete date? Or both dates?
9. Pg 3.4.2-1
a. Can we get the additional details NSF PO expects to see in the PEP?
10. Pg 4.2.1-2, NSF requires 90-180 days to do a cost analysis before we progress to the next
design phase, yet the figures 4.2.1-1 reflects this analysis is done post FDR but prior to
construction award. 1 – It sounds like NSF intends to perform this prior to FDR, is this
accurate and if yes why the deviation from the LFM. 2. Does NSF expect to do a cost
analysis of our subcontractor proposals (IAW 4.2.1) prior to award of construction? THIS IS
ANOTHER CRITICAL ITEM AND REQUIRES IMMEDIATE DISCUSSION AND RESOLUTION. (LM to
provide graphic)
PJJ Comment: It is my assumption that in order to complete a cost analysis post FDR but
prior to award of construction (CLIN 005) we will need to have our subcontractor
proposals received, down select, site visits, final proposals, orals, negotiations and LMs
analysis. After all of that, then NSF will start their analysis of the LM submittal which will
take up to 180 days. It will take LM 30 days to finalize our review and analysis, NSF 180
days to do the same. Then at least 30 more days for the NSF to issue the modification to
our prime contract and 20 days for LM to process the award once funding has been
received – this is 260 days or 8.6 months. This is an extraordinary amount of time for
construction proposal validity period. Three months is often at the high end of a validity
period range. However, extending this period will cause subcontractors to price in
additional risk due to cost of materials, resource availability, etc.
11. Pg 4.2.2-7 Section 4.2.2.4, G.5 –
a. 8/5/16: Clarification on the reference to uniform guidance, 2 CFR section #200.331 is
incorrect so needs to be fixed. GPO.gov says parts 305-399 are reserved. Would like
clarification on intent of this reference or an alternate source document for its
availability. https://www.gpo.gov/fdsys/search/pagedetails.action?collectionCode=CFR
&searchPath=Title+1%2FChapter+III%2FPart+304&granuleId=CFR-2014-title2-vol1sec200-331&packageId=CFR-2014-title2vol1&oldPath=Title+1%2FChapter+III&fromPageDetails=true&collapse=true&ycord=479.
12. Pg 4.2.2-8 Management Fees
a. 8/5/16 - This needs to be clarified for situations involving contracts. This addresses
cooperative agreements. Contracts need to be addressed differently or its specific
exclusion noted in the IMP to avoid confusion (see question 1 and Pg 2.1.3-1).
13. Pg 4.2.3-3 and Figure 4.2.3-2.
a. 8/5/16 - Are we required to provide the cross-walk to the NSF budget format/category
code?
6

Meeting Date: 8/5/2016

14. Pg 4.2.3-5 Construction Cost Book Detail and Supplementary Guidance
a. Is this how they want to see our subs proposals? Do subs need to submit BOEs for each
labor category? Do they need to use NSF budget category codes? If yes, do we want to
have the GC use A-I or just G-5 for all BOEs (this section refers back to section
4.2.2.4)? While I am not used to doing this for construction it is not impossible. It
should be noted that in addition to BOEs there will be need a detailed cost estimate by
WBS in excel to match up with each BOE.
i. 8/5/16 - Requiring a General Contractor to provide this level of detail for each labor
category seems unnecessary. This may be more appropriate for contracts that are not
fixed price.
15. Pg. 4.2.5-1 – Discussion on Budget Contingency Planning for the Construction Stage.
a. This section states, “In support of NSF’s “No-Cost Over-run” policy, confidence levels
(does “confidence level” mean high level total project cost and high level scope (buildings
in/buildings out), or detailed IMS level tasks, resources, cost spreads?) must be in the 7090% range when the project baseline is set following PDR depending on the nature of the
project; including the ability to de-scope.
i. Subcontract awards for the construction will not be finalized by PDR so requiring a 7090% confidence in our baseline at this time may be challenging. Is it up to the PO to
either de-scope or use any budget contingency if subcontractor pricing comes in
above the estimate.
16. Pg 4.5.3-2– What BSRs is NSF planning to do and when?
17. Pg 4.5.3-3 – This is a general question but is NSF considering all three buildings one award or
each building individually? Does the NSF plan to perform an incurred cost audit on any
subcontractor award that exceeds $100M?

7