AMCS Survey Supporting Statement Part B 6-8-2012

SUPPORTING STATEMENT

Survey of Airman Satisfaction with Aerospace Medical Certification Services

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

• The universe of potential respondents is listed in the table below. Student and recreational pilots are not included in the population of interest for this survey.

	Medical Certificate Class Applied For
Pilot Certificate Level	1st Class	2nd Class	3rd Class
Airline Transport Pilot	86,416	23,061	4,883
Commercial Pilot	12,303	41,729	22,922
Private Pilot	1,753	13,971	162,979
Total	100,472	78,761	190,784

• Previous administration of this survey indicated a response rate of approximately 35%

• Similar survey methods with similar populations are receiving 70% of completions online

• The table below outlines the number of surveys needed in the sample to accurately reflect the target population.

	Medical Certificate Class Applied For
Pilot Certificate Level	1st Class	2nd Class	3rd Class
Airline Transport Pilot	266	264	253
Commercial Pilot	261	265	264
Private Pilot	232	262	266
Total	759	791	783

• The total number of medical certificates applied for in each class from the table above were used to determine how many surveys would need to be distributed in order to accurately reflect the population of each class of medical certificate applied for and each pilot certificate level.

• The table below outlines the number of surveys that need to be distributed to each of the nine groups using a 95% confidence level, 6% margin of error, and assuming a 35% response rate.

	Medical Certificate Class Applied For
Pilot Certificate Level	1st Class	2nd Class	3rd Class
Airline Transport Pilot	760	754	723
Commercial Pilot	745	757	755
Private Pilot	664	749	760
Total	2168	2260	2238

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Statistical methodology for stratification and sample selection: A random stratified sampling strategy will be used in this information collection. The stratification variables will be class of medical certificate applied for by the airman (class 1, 2, or 3) and the pilot certificate level (Air Transport Pilot, Commercial Pilot, Private Pilot, or Student Pilot) as reported in the Airman Medical Certification System (AMCS).

Stratification by pilot certificate will be based on the highest level of pilot certificate held by the pilot at the time of application for a medical certificate. Stratification by medical certificate will be based on the class of certificate for which the airman applied.

Estimation procedure: The estimated sample size was developed based on the population of each cell. Sample estimates were developed to achieve a 95% confidence level and 6% margin of error.

Degree of accuracy needed: A margin of error of 6% will meet the requirements of this project. Using a 95% confidence level, the number of usable returns required to achieve that accuracy is 2,333 surveys overall.

Unusual problems requiring specialized sampling procedures: None

Any use of periodic (less than annual) data collection cycles to reduce burden: To reduce burden, the survey will be conducted in the year OMB approval is given and then again two years later.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

Methods to maximize response rates:

• The notification/invitation package will include a cover letter worded to engage the pilots’ interest, the reasons for the survey, and the benefits that they will receive from participating. The invitation package will be sent from the Office of the Federal Air Surgeon.

• We will request that pilot magazines, newsletters, websites, and organizations encourage airman participation.

• The survey will be made available in an online and hard copy version.

• Reminder post cards will be sent to sub-groups 2 weeks and 4 weeks following the initial invitation.

• At 6 weeks a letter will be sent to individuals in cells where the response rates are not adequate based on the sampling plan.

Methods to assess generalizability:

• Non-response analyses will be conducted using demographic data available in the AMCS system, including: certificate applied for, status of application, type of pilot certificate, age, gender, geographical region.

• If survey respondents differ significantly from the population, weighted sample procedures will be used to mitigate the bias.

• Characteristics of online respondents will be compared to the characteristics of the population and participants who completed the printed version of the survey to determine if there is any difference in responses by format.

• If there are differences, weighted sample procedures will be employed to mitigate the bias in subsequent analyses.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

No beta testing will be conducted as the survey has been administered before.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The survey development and analysis will be conducted under the direction of Katrina Avers, a Research Psychologist employed by the FAA at the Civil Aerospace Medical Institute, Oklahoma City (405) 954-1199 ([email protected]).

The survey will be distributed and the data will be collected by a survey contractor, Xyant Technologies, Inc. The point of contact for data collection will be Janine King ([email protected]).