2120-0707 2015 B

SUPPORTING STATEMENT

Survey of Airman Satisfaction with Aerospace Medical Certification Services

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

• The population of potential respondents is described in Table 1 below. Student and recreational pilots are not included in the population of interest for this survey.

Table 1.

	Population of Medical Certificate Groups by Application Class*
Pilot Certificate Level	1st Class	2nd Class	3rd Class
Airline Transport Pilot	86,416	23,061	4,883
Commercial Pilot	12,303	41,729	22,922
Private Pilot	1,753	13,971	162,979
Total	100,472	78,761	190,784

*Note: The size of the population changes daily as applications for certification are applied for, accepted, and rejected on a daily basis. The numbers are provided by the Aerospace Medical Certification Division and are simply provided as an estimate of the population. The proportions of each application class/pilot certificate level are typically static but the actual numbers are dynamic.

• Sample Size was determined by the following formula:

* for ease, the following online calculator was used: http://www.custominsight.com/articles/random-

sample-calculator.asp

**using a normal distribution (p = 50%)

***Rea, L. M., & Parker, R. A. (1992). Determining the Sample Size. In, L. M. Rea & R. A. Parker, Designing

and conducting survey research: A comprehensive guide.  (pp. 142-156). San Francisco: Jossey-Bass.

• Table 2 outlines the sample size needed to accurately reflect the target population assuming a normal distribution (Table1).

• Table 2 below outlines the number of surveys needed in the sample to accurately reflect the target population using a 95% confidence level and 6% margin of error.

Table 2.

	Sample Size Needed for Medical Certificate Groups by Application Class
Pilot Certificate Level	1st Class	2nd Class	3rd Class
Airline Transport Pilot	266	264	253
Commercial Pilot	261	265	264
Private Pilot	232	262	266
Total	759	791	783

• The total number of medical certificates applied for in each class from Table 2 were used to determine how many surveys would need to be distributed in order to accurately reflect the population of each class of medical certificate applied for and each pilot certificate level.

• Previous administration of this survey indicated a response rate of approximately 35%

• Therefore, Table 3 below outlines the number of surveys that need to be distributed to each of the nine groups using a 95% confidence level, 6% margin of error (Table 2), and assuming a 35% response rate.

Table 3.

	Number of Surveys to be Distributed Assuming 35% Response Rate for Medical Certificate Groups by Application Class
Pilot Certificate Level	1st Class	2nd Class	3rd Class
Airline Transport Pilot	760	754	723
Commercial Pilot	745	757	755
Private Pilot	664	749	760
Total	2168	2260	2238

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Statistical methodology for stratification and sample selection: A random stratified sampling strategy will be used in this information collection. The stratification variables will be class of medical certificate applied for by the airman (class 1, 2, or 3) and the pilot certificate level (Air Transport Pilot, Commercial Pilot, Private Pilot, or Student Pilot) as reported in the Airman Medical Certification System (AMCS).

Stratification by pilot certificate will be based on the highest level of pilot certificate held by the pilot at the time of application for a medical certificate. Stratification by medical certificate will be based on the class of certificate for which the airman applied.

Estimation procedure: The estimated sample size was developed based on the population of each cell. Sample estimates were developed to achieve a 95% confidence level and 6% margin of error.

Degree of accuracy needed: A margin of error of 6% will meet the requirements of this project. Using a 95% confidence level, the number of usable returns required to achieve that accuracy is 2,333 surveys overall.

Unusual problems requiring specialized sampling procedures: None

Any use of periodic (less than annual) data collection cycles to reduce burden: To reduce burden, the survey will be conducted in the year OMB approval is given and then again two years later.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

Methods to maximize response rates:

• The notification/invitation package will include a cover letter worded to engage the pilots’ interest, the reasons for the survey, and the benefits that they will receive from participating. The invitation package will be sent from the Office of the Federal Air Surgeon.

• We will request that pilot magazines, newsletters, websites, and organizations encourage airman participation.

• The survey will be made available in an online and hard copy version.

• Reminder post cards will be sent to sub-groups 2 weeks and 4 weeks following the initial invitation.

• At 6 weeks a letter will be sent to individuals in cells where the response rates are not adequate based on the sampling plan.

Methods to assess generalizability:

• Non-response analyses will be conducted using demographic data available in the AMCS system, including: certificate applied for, status of application, type of pilot certificate, age, gender, geographical region.

• If survey respondents differ significantly from the population, weighted sample procedures will be used to mitigate the bias.

• Characteristics of online respondents will be compared to the characteristics of the population and participants who completed the printed version of the survey to determine if there is any difference in responses by format.

• If there are differences, weighted sample procedures will be employed to mitigate the bias in subsequent analyses.

NOTE: Using this methodology, the response rate for the 2014 survey administration was 36.6%. Of 3,775 AMCS’s that responded to the invitations, 3,769 met the eligibility criteria of applying for a Class 1, Class 2, or Class 3 license.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

No beta testing will be conducted as the survey has been administered before.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The survey development and analysis will be conducted under the direction of Katrina Avers, a Research Psychologist employed by the FAA at the Civil Aerospace Medical Institute, Oklahoma City (405) 954-1199 ([email protected]).

For correspondence please contact Michelle Bryant, a Research Psychologist employed by the FAA at the Civil Aerospace Medical Institute, Oklahoma City (405) 954-9506 ([email protected]).

The survey will be distributed and the data will be collected by a survey contractor, Xyant Technologies, Inc. The point of contact for data collection will be Janine King ([email protected]).