2120-SAFE Program Survey Part B_revised

Department of Transportation

Federal Aviation Administration

Paperwork Reduction Act Supporting Statement

Aviation Safety, Flight Standards Service

Safety Awareness, Feedback, and Evaluation (SAFE) Program: Customer Satisfaction Survey

OMB Control Number 2120-XXXX

Part B: Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used.

The below table contains the universe of potential respondents. Each potential respondent, as a member of a stakeholder group, is required by federal regulations to hold a current certificate issued by the FAA Flight Standards Service. The required returns are based on a 50/50 split in the population on characteristics of interest +3 percentage points target precision and 95% confidence levels. Our experience conducting customer satisfaction surveys with aviator populations produced response rates ranging between 35-65%. Because this is the first time that respondents will be offered options in accessing the survey (i.e., personal computer, mobile device, or print), we are basing the drawn sample size on a 50% response rate, the mid-point of the range. We expect at most that 10% of respondents will request a paper version.

	Stakeholder Group
	Commercial and Airline Transport Pilots	General Aviation and Non-commercial Pilots	Repair Station Operators	Aviation Maintenance Technicians and Repairers	Air Carrier Operations Management
Population Size (2013 data)	254,863	200,215	4,801	334,694	2,213
Required Returns* (50/50 split; target precision +3 percentage points with 95% confidence)	1063	1062	873	1064	720
Drawn Sample Size (adjusted for 50% response rate)	2128^	2128^	1746	2128^	1440
^Based on 1064, the largest of the computed sample sizes for populations of 200,000+.

2. Describe the procedures for the collection of information including:

A simple random sample will be drawn from the sampling frames provided by the FAA system of records for certified pilots, mechanics, and air carriers. The SAFE program standard for precision in the sample data is +3 percentage points from the population value with a 95% confidence level that the estimate is within the margin of error.

Further, a mix-mode survey approach is being used, so respondents can choose the most convenient way to submit a survey--via a personal computer (PC), mobile device, or on paper. Thus, a unimode survey design is being used for each stakeholder survey to present respondents with identical experiences (e.g., item sequence, type, and wording) no matter which administration mode they choose.

3. Describe methods to maximize response rates and to deal with issues of non-response.

Methods to maximize response rates:

• A mixed-mode survey methodology is being used to allow respondents the convenience of choosing how (PC, mobile device, paper) and when they complete the survey, within the opening and closing dates.

• Notices will be published in trade magazines and posted on professional organizations websites, to alert potential respondents of the upcoming survey.

• An invitation from the FAA Flight Standards Service to participate will be mailed out to potential respondents, once the survey is online. The invitation will detail the purpose of the survey; the importance and benefit of providing feedback to improve service quality and delivery; directions for accessing the survey in each mode; the date the survey closes; and contact information for tech support and questions.

• Returns will be tracked based on a unique identifier; so that reminder letters from the FAA Flight Standards Service sent out 2 weeks and 4 weeks after the survey opens will only go to those potential respondents that have not submitted a survey.

• At 2 weeks before the survey closes, a final reminder letter from the FAA Flight Standards Service will be sent to those who have submitted a survey.

Methods to assess generalizability:

• Non-response bias analyses will be conducted to determine if respondents differ systematically from non-respondents. The two groups will be compared on proportions of relevant characteristics found in the sampling frame data (e.g., type of certificate, years certified, region operating out of, and size of aviation operation). Proportional differences greater than 15 percentage points will be examined in relation to impact on survey outcomes to determine if it is necessary to contact non-respondents to minimize bias. Foremost, the analysis will reveal potential limitations of the survey results.

• Comparisons of responses will be made among survey modes while controlling for respondent characteristics to test for response biases before compiling the final dataset.

• Weighted sample procedures will be used to mitigate bias if respondents differ significantly from the population.

4. Describe any tests of procedures or methods to be undertaken.

A thorough item review process will be employed to include reviews by subject matter experts to ensure proper use of terminology and comprehensiveness of response options. As well, the electronic survey modes will undergo verification and validation to ensure that the items and response options properly display and correctly respond to input, and that the output is accurately stored.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The survey design and analysis will be conducted under the direction of Katrina Avers [[email protected] or (405) 954-1199] and Brenda Wenzel [[email protected] or (405) 954-9605], Research Psychologists employed by the FAA, Civil Aerospace Medical Institute, Oklahoma City.

The survey will be distributed and the data will be collected by a survey contractor, Xyant Technologies, Inc. Their point of contact will be Janine King ([email protected]).

3