2120-0605 Supporting Statement B 10 18 final ML

Supporting Statement B

B. Statistical Methods

1. Describe the potential respondent universe.

Of a total 1562 Production Approval Holders (PAH), 621 are subject to receive a periodic Quality System Audit (QSA) conducted by the FAA. The frequency that a PAH receives a QSA is based on a cycle that repeats every twelve, thirty-six or forty-eight months depending on the complexity of the product or article produced by the PAH. This calculus is determined for each PAH through their product or article’s risk level of exposure in the national airspace system and it’s organizational risk assessment. The variation in the annual number of QSA audits conducted may vary from about 160 to about 220 depending on the continuing cycle of renewed due dates. The average is approximately 190 QSA audits in a fiscal year.

In FY 2017 there were 135 QSA audits conducted at production approval holders nationwide. Every production approval holder (PAH) subject to a Quality System Audit (QSA) is provided with the opportunity to critique their specific audit process experience by submitting the feedback survey form. The form is limited to twelve multiple choice questions, with an option to comment on each. The form is directed to and usually returned from the PAH Quality System manager or director. Although metrics are not readily available for all FY17 responses, the majority of feedback surveys are completed and returned to the FAA.

2. Describe the procedures for the collection of information.

The submittal of feedback data is strongly encouraged but not required. PAH facilities receive the feedback form by U.S. mail prior to the QSA and are reminded during the audit to complete and submit the form to their managing field office within 30 days of conclusion of the QSA. The results are reviewed by their local managing office, scanned and electronically forwarded to their appropriate manufacturing inspection office and any other relevant FAA offices for review and record keeping. Feedback forms are reviewed for any issues which provide an opportunity for continuous improvement to the QSA program and to provide the PAH an opportunity to voice an opinion regarding the professionalism, conduct, and thoroughness of the FAA audit team.

3. Describe methods to maximize response rates.

Submittal is voluntary and verbally encouraged in the briefings at the beginning and ending of each QSA.

PAH facilities receive the feedback form by U.S. mail prior to the QSA and are reminded during the audit to complete and submit the form to their managing field office within 30 days of conclusion of the QSA. The direct benefits of the feedback report are fully explained during the opening and closing briefings and that the feedback information is fully visible to personnel in several FAA managing offices

Feedback forms are reviewed for any issues which provide an opportunity for continuous improvement to the QSA program and to provide the PAH an opportunity to voice an opinion regarding the professionalism, conduct, and thoroughness of the FAA audit team. As the feedback information is directed to the managing offices of the FAA team conducting the audit, it offers FAA management the visibility and insight into the objectivity and focus of the audit from the industry perspective.

It should be noted that the significant majority of feedback responses are very positive. If suggestions for improvements are offered and deemed appropriate, the managing field office will usually determine if policy will be affected. If a policy update is in the best interest of the agency and the PAH, the managing field office will most likely submit the proposed policy change to AIR 6D2 for consideration through the automated directive feedback system. The QSA process has been in place for many years and policy updates resulting from QSA feedback have been minimal in recent years. The FAA certificate management process which includes the QSA however is evolving to adopt new methodology and the value added by the continued use of the QSA feedback survey remains valid in that context.

4. Describe tests of procedures and methods to be undertaken.

No testing of procedures are applicable.

5. Provide the names of consultants and the person who will collect and analyze the information.

The Principal Inspector of the PAH and the manager of the local managing office are the primary recipients of the QSA feedback survey. The managing office typically forwards the survey to their appropriate Manufacturing Inspection Office for their upper level review.

Although not required under policy, some managing offices enter the feedback electronically to the aircraft certification automated information system and / or send the feedback to AIR 6D2 Surveillance and Oversight Policy Section for review to determine if changes to the training agenda need to be taken into consideration.