Supporting Statement A- NTSB Training Ctr eval forms 12-2-14

National Transportation Safety Board

Generic Approval of NTSB Training Center Evaluation Forms

A. JUSTIFICATION

1. Circumstances that make the collection of information necessary.

The National Transportation Safety Board (NTSB) offers training programs to staff and the public on a variety of topics. The NTSB’s authorizing statute provides the agency is authorized to do the following:

[N]egotiate and enter into agreements with individuals and private entities and departments, agencies, and instrumentalities of the Government, State and local governments, and governments of foreign countries for the provision of facilities, accident-related and technical services or training in accident investigation theory and techniques, and require that such entities provide appropriate consideration for the reasonable costs of any facilities, goods, services, or training [the NTSB provides].

49 U.S.C. § 1113(b)(1)(I).

Training of Board employees and others.--The Board may conduct training of its employees in those subjects necessary for the proper performance of accident investigation. The Board may also authorize attendance at courses given under this subsection by other government personnel, personnel of foreign governments, and personnel from industry or otherwise who have a requirement for accident investigation training. The Board may require non-Board personnel to reimburse some or all of the training costs, and amounts so reimbursed shall be credited to the appropriation of the Board as offsetting collections.

49 U.S.C. § 1115(d)

In accordance with these authorities, the NTSB operates a Training Center in Ashburn, Virginia.

The mission of the Training Center is to promote safe transport by:

• Ensuring and improving the quality of accident investigation through critical thought, instruction, and research;

• Communicating lessons learned, fostering the exchange of new ideas and new experience, and advocating operational excellence;

• Providing a modern platform for accident reconstruction and evaluation; and

• Utilizing its high-quality training resources to facilitate family assistance and first responder programs, sister agency instruction, and other compatible federal activity.

In administering training courses designed to achieve these objectives, the NTSB seeks to maintain a standard of excellence. The NTSB’s goal of providing materials, instructors, methods of instruction, and facility arrangements that are satisfactory will require the NTSB to obtain feedback on the training courses from attendees.

Generic approval for all Training Center course evaluation forms will allow the NTSB to better obtain feedback from all attendees of courses. The NTSB will utilize this feedback to ensure it is fulfilling its objectives for each course; in this regard, the NTSB plans its course offerings based on the level of interest from potential attendees and on the degree to which attendees have found useful the information they learned during such courses. Moreover, the NTSB will tailor each evaluation form to ensure it requests feedback specific to the particular course of which the NTSB seeks evaluation. Consistent with the OIRA Administrator’s guidance concerning generic approvals,¹ the NTSB will not be able to finalize draft evaluations specific to each course until the NTSB offers the course. These types of questions are unique to the specific course, and impossible to know prior to the offering of the course.

2. How, by whom, how frequently, and for what purpose the information will be used.

The NTSB offers training courses to the public, employees of other federal agencies, and NTSB employees. Currently, the NTSB offers the following training courses, about which the NTSB seeks approval for evaluation forms: Accident Investigation Orientation (RPH301); Aircraft Accident Investigation (AS101); Aircraft Accident Investigation for Aviation Professionals (AS 301); Cognitive Interviewing Series (IM401S); Family Assistance (TDA301); Investigating Human Fatigue Factors (IM303); Managing Communications During an Aircraft Accident or Incident (PA302); Managing Communications Following a Major Transportation Accident (PA303); Managing Transportation Mass Fatality Incidents (TDA406); Marine Accident Investigation (MS101); Mass Fatality Incidents for Medicolegal Professionals (TDA403); Rotorcraft Accident Investigation (AS103); Accident Site Photography (IM300S), Rail Transit Party Process (RPH304), Survival Factors in Aviation Accidents (AS302), and Weekend Safety Seminars. In response to attendee feedback, requests for training in specific areas, and other considerations, the NTSB will likely add or remove classes from this list in the coming years. Therefore, the listing in this paragraph is not exhaustive; hence, this proposed information collection is appropriate for generic approval.

Staff at the NTSB Training Center plan to offer course evaluation forms to each attendee of each course. The NTSB offers 12 different courses per year and provides a repeated program for those courses with the highest attendance levels. Among all courses, the NTSB estimates a total of 600 non-Government attendees complete courses in any given year. As a result, the NTSB estimates it will distribute approximately 600 Training Center evaluation forms each year. The courses listed above are provided at least once a year. The NTSB may offer some courses more than once per year. Each evaluation form will take approximately 11 minutes to complete.

At the conclusion of each course, the NTSB seeks to offer a paper copy of an evaluation form and encourage attendees’ completion of the form. The NTSB may plan to contact attendees following their departure from the Training Center, in order to obtain completion of the form and/or follow up on any responses about which the NTSB may seek additional information. In such communication, the NTSB may use e-mail or a commercially available survey methodology and/or telephone to correspond with attendees. The Training Center will also use electronic version of the evaluation.

This type of information collection is appropriate for generic approval under the OIRA Administrator’s guidance. The NTSB periodically changes the identification numbers and subject matter addressed in NTSB training courses. Such variance renders generic approval appropriate. By distributing evaluation forms, the NTSB will gather feedback concerning whether attendees found the instructor knowledgeable and helpful; whether the course materials were appropriate; the location and course facilities; the “case studies” discussed in the course; and other similar topics. Each course evaluation form will include some course-specific questions. Responses to such evaluations will assist the NTSB in ensuring its courses work to fulfill the goals listed above.

Evaluations of NTSB Training Center courses will influence future course content and offerings. The NTSB will rely upon the provision of completed course evaluations to assist with the planning of course offerings. The NTSB’s desire to obtain information immediately following a training course will assist the NTSB Training Center in developing courses to achieve the NTSB’s objective of improving investigators’ and transportation industry peers’ accident investigation practices and techniques. Immediate responses are also critical to the NTSB’s evaluation of courses because attendees’ memories may fade over time. In addition, response rates would most likely decline if the NTSB needed to wait for the normal approval process under the Paperwork Reduction Act. Overall, the NTSB believes generic approval for Training Center course evaluation forms is appropriate, based on the circumstances and the NTSB’s goals.

3. Use of automated, electronic, mechanical, or other technological techniques or other forms of information technology.

At the conclusion of each course, the instructor or an NTSB employee will hand out evaluation forms and encourage attendees to complete them. The instructor or employee will instruct attendees to complete the form and hand it in before departing the facility. However, the instructor or NTSB employee will also provide an e-mail address, phone, fax, and a postal mail address by which a respondent may send his or her completed form. The NTSB employee may also notify the attendee using a commercially available survey program depending on the method of response that is most available and comfortable for the attendee.

4. Duplication.

The NTSB’s course evaluation forms are tailored for each individual course. The NTSB provides a unique slate of transportation event investigation training. While other agencies offer investigative courses, they are dissimilar to those the NTSB offers. Any course evaluation forms other agencies may offer would not be duplicative of the NTSB’s course evaluation forms. As a result, the NTSB has not identified a likelihood of any duplication.

5. Small businesses or other small entities.

The NTSB will distribute the form to individuals who personally attended an NTSB Training Center course. The NTSB does not anticipate its solicitation of feedback via the form will create a significant burden for any small business or entity, because the questionnaires will take no more than approximately 11 minutes to complete; beyond completion of the form, the NTSB will not require any recordkeeping or other similar activities.

6. Consequences to the Federal program or policy activities if the collection is not conducted or is conducted less frequently.

The NTSB Training Center needs to obtain information from course attendees in order to ascertain several aspects of each course and evaluate whether staff at the NTSB Training Center should make changes to courses. For example, the NTSB will need feedback concerning the effectiveness of the course content and materials, as well as the suitability of the course instructor. Such feedback will assist with the NTSB’s fulfillment of its statutory authorities codified at 49 U.S.C. §§ 1113(b)(1)(I) and 1115(d)

.

7. Special circumstances that require the collection to be conducted in a manner inconsistent with OMB guidelines.

As explained above, the NTSB is requesting generic approval for its Training Center course evaluation forms for which it will request feedback from each course attendee. Other than the request for generic approval, no special circumstances exist. The collection of information will be conducted in a manner consistent with the guidelines in 5 C.F.R. § 1320.5(d)(2).

8. Federal Register publication.

The NTSB published two notices in the Federal Register concerning this collection of information. See 79 FR 14302-02 (March 13, 2014) and 79 Fed. Reg. 61342 (October 10, 2014). The NTSB did not receive any comments concerning the proposed plan for generic information collection.

9. Gifts or payments to respondents.

The NTSB will not provide any gifts or payments to respondents for completion of course evaluation forms.

10. Assurance of confidentiality.

The NTSB will not provide any assurance of confidentiality to respondents concerning the completed questionnaires. The NTSB will encourage respondents to refrain from including their names on the course evaluation forms, and will not attempt to identify the respondents. Each form the NTSB distributes will explain this practice in the “Instructions” section. All subsequent correspondence with the attendee will clearly delineate the assurance of confidentiality to the attendee and state that personal identification is neither requested nor preferred.

11. Additional justification for questions of a sensitive nature.

No NTSB Training Center course evaluation form will solicit any information of a sensitive nature.

12. Estimate in hours of the burden of the collection of information.

• The NTSB may request, at most, responses from approximately 600 course attendees per year who are not government employees.

• Completion of the form is voluntary. The NTSB anticipates receiving approximately 50 percent of questionnaires completed.

• The NTSB estimates completion of the form will take approximately 11 minutes or less.

• Based on the estimate of approximately 600 attendees per year who may receive a questionnaire, fifty percent of whom may return the questionnaire, the NTSB estimates the sum of annual reporting burden hours to be 55 per year, at most.

13. Estimate of the total annual cost burden to the respondents or record-keepers resulting from the collection.

Completion of the form does not require any record keeping, capital, start-up, or maintenance costs. Instead, completion of the form will only require approximately 11 minutes of each respondent’s time, as described above.

14. Estimates of annualized cost to the Federal government.

The NTSB incurs costs in transmitting and collecting the questionnaires, as well as costs in handling and reviewing the information after the NTSB receives completed questionnaires. In transmitting the questionnaire to respondents, the NTSB opts to utilize paper copies, but the NTSB may, in some cases, send the forms to attendees via e-mail attachment. The NTSB estimates the total cost of NTSB employees’ initial intake of the form will be $40,956 per year. This total is based on an average estimate of providing 600 forms per year, with each NTSB employee expending 1.0 hours per form at an hourly rate $68.26. In estimating 1.0 hour per course, the NTSB has considered employees’ time in all of the following duties: proofreading the forms; printing the form with a high-speed printer; notifying the respondents and explaining the purpose of the form; and receiving the completed form and filing or sharing it with the appropriate NTSB employees and/or instructors.

Second, following the collection of all completed forms, NTSB staff will spend approximately 1.5 hours per year, at a salary rate of approximately $68.26 per hour, evaluating the feedback on the evaluation forms and determining and/or recommending any changes with regard to course offerings, content, materials, and instructors. Based on the estimate of receipt of 300 forms per year, the cost of NTSB employees’ review of the forms will total approximately $30,717 per year.

In sum, the NTSB’s estimate for the complete annualized cost to the Federal government totals $71,673.

15. Program changes or adjustments.

No changes or adjustments will occur to any programs.

16. Plans for tabulation and publication of responses.

Information and data the NTSB obtains in response to the forms will be valuable to the NTSB, insofar as it will ensure the NTSB is fulfilling an objective provided in the NTSB’s enabling legislation. 49 U.S.C. §§ 1113(b)(1)(I) and 1115(d). The NTSB may include some data from completed course evaluations in its Annual Report to Congress, the submission of which 49 U.S.C. § 1117 requires. For example, the NTSB may cite how many respondents indicated courses were helpful, and how the NTSB Training Center is acting on such feedback.

OPM’s Training Evaluation Field Guide, January 2011,² states training evaluation is an objective summary of quantitative and qualitative data gathered about the effectiveness of training. The primary purpose of evaluation is to make good decisions about use of organizational resources. Training evaluation data helps the organization to determine whether training is accomplishing its goals and contributing to the agency mission. It also helps agencies decide how to adjust the training and other interventions for greater effectiveness.

Evaluation data enable judgments about the following questions:

• How well did the training meet the development needs identified?

• How well did the learners master the training content?

• How well did the learning transfer to the work setting?

• How well did the training contribute to the achievement of the agency’s mission?

(From OPM’s Evaluating Training: A Primer, 2005³)

17. Display of expiration date.

For this generic collection, on each form, the NTSB will display the expiration date of OMB approval. The NTSB does not request approval to refrain from displaying the expiration date of OMB approval for the questionnaires.

18. Exception to certification statement in Form 83-I.

The NTSB does not request any exception to the certification statement contained in Item 19 of OMB Form 83-I.

1 Paperwork Reduction Act—Generic Clearances, available at http://www.whitehouse.gov/sites/default/files/omb/assets/inforeg/PRA_Gen_ICRs_5-28-2010.pdf.

2 http://www.opm.gov/policy-data-oversight/training-and-development/reference-materials/training_evaluation.pdf.

3 https://www.opm.gov/policy-data-oversight/human-capital-management/reference-materials/leadership-knowledge-management/evaluatingtraining.pdf

5