MEP Center Performance Evaluation

NIST

National Institute of

MEP - MANUFACTURIMG Standa7ds'and'TecSogy
EXTENSION PARTNERSHIP^ U.S. Department of Commerce

1 MEP Center Performance Evaluation
2

3
MEPP
102.01
4 Effective Date: 05/14/2018
5 PURPOSE
6 This policy provides guidance for evaluating the performance ofNIST MEP-funded Centers and
7 the overall program through a Program Evaluation.

8 SCOPE
9 This policy is applicable to the Hollings Manufacturing Extension Partnership Program and all
10 MEP Center cooperative agreements awarded by NIST under the base MEP Center Program (15

11 U.S.C. § 278k). It does not apply to awards issued pursuant to the MEP Competitive Awards
12 Program (15 U.S.C. § 278k-l) or Assistance to State Technology Programs (15 U.S.C. § 2781).

13 LEGAL AUTHORITY AND REFERENCES
14 • 15 U.S.C. § 278k, Hollings Manufacturing Extension Partnership, as may be amended
15 • 15 C.F.R. § 290.8, Reviews of Centers
16 • U.S. Department of Commerce Financial Assistance Standard Terms and Conditions,
17 March 31, 2017, as may be amended
18 • MEP General Terms and Conditions, August 2017, as may be amended
19 • MEP Policy P102.02, Program Perfonnance Measurement

20 • MEP Policy P102.03, Center Probation

21 DEFINITIONS
22 • Performance - Delivered impacts, measured via the 10 IMPACT survey-based
23 components, and multi-Center cooperation/collaboration measures (behavioral factors),

24 as required by 15. U.S.C. §§ 278k(g)(3)(B) and 278k(f)(2)(C).
25 • IMPACT - Improving Manufacturing Productivity and Competitiveness Tracker, the
26 scorecard by which NIST MEP measures performance.
27 • Annual Review - A review process conducted by NIST MEP during each year of a
28 Center s cooperative agreement. Annual reviews may address issues other than
29 performance if warranted, and are used to track progress of a Center against the goals of

30 any success plan. In conjunction with the MEP Grants Officer, NIST MEP may initiative

MEP Policy 102.01 Ver 1.0 [Uncontrolled Copy in Print) Page 1

31 appropriate remedial and/or enforcement actions as a result of deficiencies identified
32 during an Annual Review.
33 • Panel Review - Statutorily required peer-review process by which NIST MEP evaluates a
34 Center's performance during the third and eighth years of a Center's operations, using an

35 evaluation panel, as defined in 15 U.S.C. § 278k(g)(l), and measuring performance as
36 defined in 15 U.S.C. § 278k(g)(3). A Center may be placed on probation (see below) and
37 may ultimately become ineligible to receive funding as a result of an "other-than38 positive" Panel Review in conjunction with failure to meet other Program requirements
39 based on uniform and fair factors.

40 • Secretarial Evaluation - Statutorily required process by which NIST MEP evaluates a
41 Center's performance during the fifth year of a Center's operations, as defined m 15

42 U.S.C. § 278k(g)(2) and measuring performance as defined in 15 U.S.C. § 278k(g)(3). A
43 Center may be placed on probation (see below) and may ultimately become ineligible to
44 receive funding as a result of an other -than-positive" Secretarial Evaluation.
45 • Performance Deficiency - Negative gap in impact area(s) sufficient to warrant immediate
46 remedial action and a failure to meet other Program requirements based on uniform and
47 fair factors.

48 • Probation - As described in 15 U.S.C. § 278g(5)(A), a status for an MEP Center resulting
49 from an "other-than-positive" review and identification of performance deficiencies by
50 either an evaluation panel or Secretarial evaluation. Probation will end when the Center's
51 re-evaluation is completed (no later than 12 months from the date of notice of probation),

52 and the Center has either remedied the cause(s) of the deficiency(ies) or is unable to do
53

so.

54 • Performance Improvement Plan (PIP) - An improvement plan that will be required of an

55 MEP Center if performance deficiencies are identified by an evaluation panel (Panel
56 Review) or Secretarial evaluation. The PLP will include specific information about the
57 remedial action, how and by whom the remedial action will be delivered and monitored;
58 and what is expected of the Center. The PIP should be incorporated into a Center s

59 cooperative agreement through a Special Award Condition(s) (SAG) issued by the NIST
60 Grants Officer. The duration of the SAG will be specified in the SAC itself. A
61 reevaluation of the Center's performance will be completed within 12 months after the
62 notice of probation to the Center whichever comes first. Note that the probationary term

63 may be shorter if the Center can rectify the deficiencies in a shorter time period.

64 POLICY
65 In accordance with the MEP statute at 15 U.S.C. § 278k, NIST MEP expends funds for the
66 creation and support ofMEP Centers through cooperative agreements with eligible entities.

67 Receipt of those funds is contingent upon both a successful application for the funds and the
68 results of evaluations conducted by evaluation panels during the third and eighth years of
69 operation and a Secretarial evaluation during the fifth year of operations. See NIST MEP
MEP Policy 102.01 Ver. 1 [Uncontrolled Copy in Print) Page 2

70 Performance Evaluation Management Policy, P 102.02. The following sections describe how
71 Program Evaluation shall take place.
72 Panel Reviews - Panel Reviews shall be performance-based peer reviews. Panels shall
73 provide feedback on Center strengths and opportunities for improvement, including areas of
74 deficiency, if any, as defined in the NIST MEP Program Performance Measurement PoUcy.
75 Panels are solely focused on performance and the factors that contribute to that performance
76 and each Panel's report will include evaluation and feedback on the adequacy of a Center's
77 Performance and Evaluation Management System, and its actual use, to support ongoing self78 assessment and performance improvement.

79
80 Performance measures used for feedback shall be the 10 IMPACT metrics, which are based
81 on survey aud Center data. Panel Review evaluations and diagnoses shall be supported by
82 NIST MEP analytical data on the Center and by overall Network performance data. Centers
83 shall have access to all input data and analyses given to its Panel.

84
85 Inputs to Evaluation Panel Reviews shall be comprised of six key documents:

86
87 1. Center Performance & Profile Report
88 2. Center Strategic Plan
89 3. Prior Year Annual Review Reports
90 4. Center Response to Pre-Panel Questions
91 5. Center Panel Review Presentation (Focused around their Performance & Evaluation
92 Management System, Center History and Key Objectives from Strategic Plan).
93 6. Center Self-Assessment

94
95 Items 1, 2, 4, 5, and 6 are provided by the Center in advance of the review.

96
97 Panel Review evaluations shall rely primarily on analysis of performance trends and
98 comparisons. To ensure that comparisons are properly capacity-scaled to total funding per

99 SMM, NIST MEP data and analyses shall explore and offer relevant additional operating
100 ratios.

101
102 Secretarial Evaluation - Secretarial Evaluations shall be perfonnance-based reviews,
103 conducted by NIST MEP's Director and Deputy Director. Performance areas used shall be the 10
104 IMPACT metrics, which are based on survey and Center data. The evaluation documentation
105 will include, at a minimum, all prior-year annual reviews, 3 year Panel Review, IMPACT
106 Performance and Trend Analysis, any success plans and results/progress related to those plans,
107 Performance Improvement Plan (if any), and an analysis of performance by the relevant NIST

108 MEP Regional Teams (Regional Managers [RM], Federal Program Officers [FPO], and others as
109 needed).
MEP Policy 102.01 Ver. 1 ^Uncontrolled Copy in Print] Page 3

110 Annual Reviews - Annual Reviews, while not statutorily required, are required in NIST
Ill MEP's regulation at 15 C.F.R. § 290.8, and are conducted each year during the life of the
112 cooperative agreement. These programmatic evaluations, aligned with the performance-based
113 evaluations, are conducted by NIST MEP Regional Teams. Data, analysis, and performance
114 reported by the Regional Managers in Annual Reviews shall ensure measurement, trend and
115 comparison consistency across Panel Reviews and Annual Reviews.

116
117 Panelists and Panel Chairs - Panelists and Panel Chairs shall, in accordance with 15 U.S.C. §

|ll8 278k(g)(l), be comprised ofNIST staff as Panel Chairs and private experts, including -Center
|ll9 peers^ and federal officials^as the panelists. Panelists and Panel Chairs shall take part m trammg,
120 covering all policy and Panel Review operational requirements, processes, and procedures. The

121 training curriculum shall cover and explain: Center and NIST MEP inputs; data analysis;
122 diagnostics; trend analysis; Center comparisons; deficiency deliberation; feedback; and key
123 details of Panel Review stages and processes. Private experts shall not be affiliated with or in
124 any way connected to the Center that is being evaluated.

125
126 Performance Deficiencies - In the Panel Review, the Panel will identify deficiencies in
127 performance, based on the same uniform and fair factors provided each Center. This includes

128 specific activities supporting the integration of the national network which are part of FPO and
129 RM input in annual reviews. Panel Review feedback regarding deficiencies found, as identified

130 by the panelists individually, and behavioral factors agreed upon within the cooperative
131 agreement, shall be included in the report to the Center, advising it of the decision to give the
132 Center an evaluation that is not positive, thereby placing the Center on probation.

133
134 Probation -~ In accordance with 15 U.S.C. § 278k(g)(5), a Center that does not have a

135 positive evaluation shall be placed on probation, beginning on the date that the Center
136 receives notice and ending on the date that the reevaluation is complete and the result of
137 the reevaluation is that the Center has corrected the identified deficiencies. The Center
138 shall also address any additional concerns with behavioral factors outlined in the

139 Performance Improvement Plan (PIP) that is required as a result of being placed on
140 probation. See NIST MEP Policy MEP Center Probation, P102.03.
141
142 If a Center receives an evaluation that is other than positive, they shall be notified of the
143 reason(s) which shall include any deficiencies in performance and concerns about behavioral
144 factors. Additionally, a Center Performance Improvement Plan, designed in consultation with
145 the MEP Director and Deputy Director, shall provide focused remedial assistance to address
146 center performance deficiencies. Specific network experts m those focused areas shall be
147 assigned to work with the center. Quarterly monitoring of the Center shall be conducted by the
148 regional management teams to document progress. A re-evaluation of the Center's
149 performance shall be completed not later than 12 months after the notice of probation to the
MEP Policy 102.01 Ver. 1 [Uncontrolled Copy in Print] Page 4

150

Center. Should a Center not rectify the stated deficiencies or show significant improvement in

151

performance before the end of the probation period, consistent with 15 U.S.C. § 278k(g)(5),

152

NIST MEP shall conduct a competition to select a new operator of the center.

153
154
155
156
157

Carroll A. Thomas

158

Director

MEP Policy 102.01 Ver.l

^Uncontrolled Copy in Print)

Date

Page 5