NIST OWM Training Evaluation Supporting Statement

OMB Control No. # 0693-0033 – NIST Generic Clearance for Program Evaluation Data
Collections
Office of Weights and Measures Training Evaluation Survey

FOUR STANDARD SURVEY QUESTIONS

1. Explain who will be surveyed and why the group is appropriate to survey.
The Office of Weights and Measures (OWM) conducts technical training, seminars, webinars
and presentations at NIST and around the country for a wide variety of customers and
stakeholders in the weights and measures community. The content and instructional methods
are technical in nature and unique for each learning event. The training events are offered to
fulfill the OWM’s mission to secure uniformity in weights and measures laws and calibration and
test procedures.
During the planning phase and upon completion, each learning event is evaluated to analyze
weights and measures training needs, obtain input from the target community, design and
deliver training for laboratory metrologists and weights and measures officials, and measure the
impact and effectiveness of training to ensure ongoing continual improvement. Through followup surveys, OWM measures students’ application of skills learned and the impact of training,
and uses the information to provide additional professional development opportunities.
2. Explain how the survey was developed including consultation with interested
parties, pre-testing, and responses to suggestions for improvement.
The survey questions were developed using the Donald Kirkpatrick Model1 that was first
presented in 1959. Questions for this survey were developed by conducting a number of focus
group meetings, and by reviewing past surveys to determine what information would be most
useful to us in identifying where changes are needed to better meet the needs of our customers.
The OWM “Procedure for Managing Customer Feedback” requires a plan be in place regarding
the gathering of data and includes the “Plan for Gathering Customer Feedback – Form.” The
learning event evaluation is comprehensive and includes all major elements such as: instructors,
logistics, learning event materials, media, technology, currency, relevance, and effectiveness.
There is also an “Analysis Report Form” available for use in studying the data. The OWM believes
the revised survey is a significant improvement over the past customer satisfaction survey.

1 the Donald Kirkpatrick four-point model 1: Reaction - to what degree participants react favorably

to the training; 2: Learning - to what degree participants acquire the intended knowledge, skills,
attitudes, confidence and commitment based on their participation in a training event; 3. Behavior
- to what degree participants apply what they learned during training when they are back on the
job; 4: Results - to what degree targeted outcomes occur as a result of the training event and
subsequent reinforcement. See www.kirkpatrickpartners.com.
NIST Weights and Measures

2

CD 1004E3-February 2016

3. Explain how the survey will be conducted, how customers will be sampled if fewer
than all customers will be surveyed, expected response rate, and actions your
agency plans to take to improve the response rate.
All students registered for a training class will be surveyed electronically or in person immediately
following the class. We expect an 80 % response rate based on interest shown by customers in
gathering this type of data. 50 % of initially responding students will be contacted a second time
to conduct a 45-day follow-up survey to collect additional feedback on how they applied the skills
learned during training. Each survey should take about 5 minutes to complete. The OWM will
promote participation in training evaluation during classes by presenting previously collected
information and highlighting the usefulness of the data for both the stakeholders and for the
continuous improvement of the program.
4. Describe how the results of the survey will be analyzed and used to generalize the
results to the entire customer population.
The results of individual training evaluations will be used by OWM staff in after-action reviews to
improve aspects of specific training programs. The purpose of collecting customer feedback is to
obtain information to improve program operations, including measuring progress towards goals,
conducting a program evaluation, incorporating continuous improvement, and measuring
performance. Specific customer complaints or negative feedback are addressed and documented
and kept on file electronically (in Survey Monkey and in the OWM Training Records). The results
are also compiled at mid-year and end-of-year for reporting success measures as stated in the
OWM’s operations plan.

NIST Weights and Measures

2

CD 1004E3-February 2016