B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.
The population of potential respondents consists of all international air passengers who are traveling on participating airlines whose trip originates in the United States or includes the United States in their itinerary. There are essentially two separate populations: non-U.S. residents inbound to the U.S. and U.S. residents outbound to overseas or Mexico.
The sample is designed around the geographic detail desired for the resulting estimates and the specific airlines willing to participate in the survey. The design is a stratified, two stage cluster sample, where scheduled flights are randomly selected from strata defined by airline and foreign destination in the first stage. The responding travelers on each flight constitute the second stage of the sample. When the SIAT is conducted on a selected flight, those passengers who respond, are considered to represent all passengers on that flight.
The development of the sample was influenced by the number of travelers to and from the United States by country of residence, area visited, and scheduled international air carrier.
The design was also influenced by the desired accuracy and detail of the resulting estimates, the airlines' willingness to participate in the survey, the availability of a sample frame, and the costs of the survey.
Stratification is used to ensure that all participating airlines and residents of countries of interest appear in the sample. In the case of foreign flag carriers, stratification by airline alone is sufficient in most cases, since they tend to serve mainly residents of their native countries and U.S. residents on flights to the United States. U.S. carriers, serving several markets of interest, are stratified by carrier and foreign destination.
Flights are selected within each stratum by simple random sampling through the use of a random number generator. Since a random sample of passengers on a flight would be difficult for the flight crews to implement, the second-stage sample includes all passengers on a given flight who respond to the survey.
The sample frame (list from which the sample is drawn) is the monthly Official Airline Guide roster of scheduled flights departing the United States on the airlines participating in the survey. Although quarterly estimates are the goal, sampling is performed monthly to distribute the sample over each quarter. Flights to be sampled are limited to those occurring during the week, beginning with the third Monday of the month.
Sampling flights one week out of the month simplifies the operating procedures for the airlines and facilitates the processing of the survey materials.
The number of flights to be sampled from each stratum were determined on a per stratum basis, with consideration given to the number of flights, number of passengers carried, foreign destinations, number of participating U.S. and foreign carriers serving the area, and airline cooperation.
In practice, some departures from the original sample design are necessary. Substitution for sampled flights is permitted in some circumstances. It has been acceptable, for example, for an airline to change the day of the flight to be surveyed if circumstances prevent the survey's execution on the sampled day.
For various reasons, some airlines occasionally do not participate for the entire quarter; so their samples are not distributed over all months of the quarter. In some cases, strata are eliminated from the sample because of difficulties by the airline in conducting the survey on the desired routes.
2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.
Design and Procedures for Information Collection:
a) Statistical methodology for stratification and sample selection:
Refer to B‑1 above, Potential Respondent Universe and Sample Design.
b) Estimation Procedure:
The primary data sources for computing estimates are the SIAT responses. Information developed by the OTTI’s U.S. International Air Travel Statistics series from the Department of Homeland Security (DHS) Forms I‑92 flight reports and I‑94 forms are also used. These reports are collected for each flight arriving in or leaving the United States, giving the number of U.S. citizens and non‑U.S. citizens on the flight. These sources provide the input to the weighted ratio estimation procedures which expand the sample information to represent all visitors to the United States. OTTI has been tracking the response rates to the survey on a monthly basis.
They are tracked two ways: airline response rates and passenger responses rates. The airline response rates, from the In-Flight method, have ranged from 52 percent in 2001 to a low of 32 percent in 2007. This illustrates the challenges of engaging the airline industry’s cooperation since 2001 (Ironically, although more carriers are participating the larger carrier cooperation has been dropping). Overall, however, when including collections made in the airport gate departure areas the average passenger response rate was 52 percent in 2007. OTTI also tracks the response to each question on the questionnaire and the breakout specifically for the spending question. These analyses show changes in individual questions which will be used the next time OTTI revises the survey instrument. OTTI has seen between 33 percent to 67 percent increases in the responses to spending questions for total trip spending, fare expenditures, and expenditures within the United States for the inbound data.
Estimation and Reliability of Results for U.S. Residents (Outbound)
The SIAT responses are the primary data source for computing estimates.
The SIAT responses provide information on distributions of variables and relationships among survey items as well as information relating the country of debarkation to the residence of the passenger. The DHS I-92 data provide total passenger volumes by port of debarkation and the number of U.S. and non-U.S. citizens.
A weight is calculated for each survey respondent. It is defined as the number of passengers, departing from the United States via scheduled international air carriers, that is represented by the respondent. Calculation of the weight is a multi-step process.
a. The initial weight of a respondent is one, unless children are part of his or her travel party.
In this case, the initial weight has a value greater than one, depending on the number of children and the size of the travel party.
b. Although 100 percent sampling is performed on a flight, there are usually some non-responses. The respondents are considered a random sample of the passengers, and each weight is increased to cover non-responses on the flight.
c. Each weight of a respondent in a stratum is increased to represent all travelers on all flights on the stratum.
d. The I-92 data are incorporated into the weights by port of debarkation to represent not only the participating, but also the nonparticipating, airlines in the survey.
Estimation and Reliability of Results for Non-Residents (Inbound)
The survey responses are the primary data source for computing estimates. Information developed from the DHS I-94 reports is also used.
The survey responses provide information on distributions of variables and relationships among survey items as well as specifics relating country of residence and port of customs of the respondent. The DHS I-94 data provide similar information for country of residence and port of customs.
A weight is calculated for each survey respondent. It is defined as the number of passengers, departing from the United States via scheduled international air carriers, that is represented by the respondent. Calculation of the weight is a multi-step process.
a. The initial weight of a respondent is one, unless children are part of his or her travel party, in which case, the initial weight has a value greater than one, depending on the number of children and the size of the travel party.
b. Both the I-94 data and survey responses are sorted and summarized by country of residence and port of customs information.
c. The weight computed for individual survey responses is the result of directly proportioning the I-94 data to the surveys.
d. The weights determined by the limiting variables in the survey responses match the corresponding control totals from the I-94 data summarized in the same manner.
The weights are then used in standard weighted ratio estimation formulas for calculating the distributions, means, and medians found in the published tables.
Because of the multistage nature of the sample design and the resulting computational burden, sampling variability has not been calculated for the estimates. Instead, the reliability of a set of related estimates is indicated by the number of respondents to the relevant questionnaire items. The more respondents, the more reliable the estimate. Judgment must be used in deciding on the degree of confidence to place in an estimate, and in its proper use. Likewise, non-sampling (response and processing) errors have not been estimated but are likely to be significant, especially when combined with sampling variability. Response errors may be of particular significance because of inaccuracies arising from language translations and currency conversions.
A low number of respondents for a quarterly estimate can cause severe distortion because of the large influence any one respondent exerts on the overall value of the estimate.
CIC Research, Inc., the contractor for the SIAT, will be estimating the effects of sampling variability and non‑response errors. These two issues would help accurately reflect the reliability and validity levels of data produced. Non‑response bias in the departure lounge survey method will be monitored by CIC Research, Inc.
c) Degree of accuracy:
The principal development and planning phases for the survey have been completed by Transportation System Center (TSC). Weighting and estimation refinements were made by Response Analysis, and CIC Research, Inc. The computation of reliability and validity levels can be developed. This will add to the cost of the survey.
d) Questionnaire Content and Design:
Distribution of the survey instrument is conducted using two methods. The first is the in‑flight method. The other method is to distribute and collect the questionnaires in boarding areas by field survey sub-contractors. Duplication is not possible since data collection would have to be by one method or another. See parts 3 and 4 of this section.
The questionnaire is distributed and collected in boarding areas. It requests international air travelers departing the United States to provide information on their activities, expenditures, travel and demographic characteristics while traveling. The questionnaire development was guided by the normal standards of questionnaire design to encourage the maximum response by the surveyed passengers. The questions are stated as simply and clearly as possible, and definitions of possibly confusing terms are provided on the forms. To reach the majority of non-English speaking travelers, the questionnaire was translated into eleven additional languages (Australia, Brazil, France, Germany, Italy, Japan, Mexico, Netherlands, South Korea, United Kingdom, and Venezuela).
In printed form, there is an English only version and ten versions with the English version on the first half followed by the foreign language version. An announcement at the top of each form tells the respondent of the availability of the other versions. Thus, the questionnaire has been designed to minimize the language obstacle that might discourage a passenger from responding. Both resident and non‑resident questions are included in the one survey instrument.
The language versions added were a requirement of the foreign flag carrier's entry into the program. Without it, the airlines felt they would not obtain a representative sample of their passengers.
The questionnaire design facilitates easy distribution and collection by eliminating the necessity for the field contractor to determine the citizenship of the passengers. They need only give every adult passenger in the boarding area a form. Likewise, the in-flight survey method is facilitated since the flight attendants are not required to determine the passenger’s citizenship. Response to this survey is dependent on the flight attendants ability to distribute and collect the questionnaires in a timely fashion.
e) Geographic Area Structure:
The geographic area structure for this survey provides a sufficient level of detail for analyzing the passenger traffic between regions of the United States, and the principal regions and countries of the world. The structure represents a compromise that must be made if reliable statistics are to be produced. The compromise is the level of geographic detail and the cost and sample size required for the survey. Any additional countries to be covered would require the addition of airlines, flights, and passengers to be surveyed, or a reduction in the sample sizes of those already covered. The eight world regions are based on those used in the U.S. International Air Travel Statistics Program produced by OTTI.
The primary geographic units of analysis for the United States will be the eleven modified Bureau of the Census Regions. Statistics are also developed for states, large metropolitan areas, and selected major tourist attractions to the extent that response frequencies will permit. OTTI’s ability to provide estimates is largely dependent upon the total number of respondents, which changes based upon the funding level obtained for the program.
Mexico is treated separately in reports on: “Mexican Travelers to the U.S.” and “U.S. Travelers to Mexico.” Similar breakouts are provided for the ports, cities/states and destinations as in the overseas reports.
f) Airline and Airport Participation:
Analyses provide information on the total and regional coverage of air traffic between the United States. The I-92 data has been used to help identify airlines to approach for inclusion into the SIAT. It will also be used in the future to identify the target carriers approached to join the survey. OTTI uses this data to identify carriers generating U.S. citizen and alien travel to and from this country. This information is critical to the success of the SIAT, because data from passengers on these participating airlines are weighted to be representative of all regional passenger traffic. This analysis could help OTTI improve our regional representativeness in the United States, and abroad. Carriers that have added new ports to their route structure are now surveyed as they are added to ensure the program remains representative of world travel patterns. OTTI also hopes to analyze this data by airline for each port, and then overall for the airline participation as well. Hopefully, these additional analyses will help identify carriers and ports where we need to focus our efforts.
Another methodology used is to conduct surveys at the airport in the boarding area. The boarding area methodology is dictated by a few airlines who insist on the boarding area survey methodology. So, to strengthen the survey by providing additional regional coverage, and to obtain access to these passengers to keep the survey representative, we have been using the more costly boarding area survey method. This methodology is also used to facilitate individual airline participation and improve airline response rates, and to ensure we maintain global coverage. An in-depth discussion of the boarding area method versus the SIAT method is provided for in the Request for Information (RFI). Go to:
http://www.tinet.ita.doc.gov/research/programs/ifs/FINAL%20RFI%20Report.htm
By selectively using the boarding area survey, important airlines that would not otherwise participate in the In‑Flight Survey are included. Currently, we survey at: Dallas, Houston, JFK, Miami, Newark, San Francisco, Sanford and Guam's International Airports. OTTI has over time tried to convince each of the airlines to switch to the less costly in-flight survey method. In the cases where our contractor has been able to develop a personal rapport with a decision-maker at each airline, we have been able to switch them to the preferred in-flight survey methodology. The rapport is only developed through personal visits that included follow-up consultations to ensure the survey meets their needs. Since money has not always been available for this purpose, we have had limited success.
3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.
OTTI has historically worked to improve response rates as resources allowed. We have also developed a number of innovative approaches to improving the cooperation that have proven successful when funded.
Passenger Response Rates:
Individuals are not required to complete the survey, and, as with all surveys, there will be refusals. In addition, although the survey is available in eleven languages, there will be passengers who will not be able to respond due to language differences. OTTI hopes to reduce this shortcoming by offering other language translations in the future. Furthermore, non‑participation is a possibility if the flight happens to be short in duration or at night when passengers are sleeping. Other ideas to improve passenger response rates have been developed, but a funding shortage has prevented implementation.
The airlines are a second area of weakness in the system for collecting completed surveys. Here, airlines staff have contributed up to 2,500 hours of time per year to distribute, collect and send surveys to the contractor. Again, airlines staff are not contractually required to distribute the surveys. Problems associated with the voluntary distribution of the surveys by airline personnel include: flight attendants are too busy and unable to pass out all or any of the questionnaires; flight attendant unions’ resistance to conducting in-flight surveys; the surveys could be distributed but not collected; the collected surveys may not be returned to the contractor; flight packages can be misplaced or lost (via the mail or by the airline personnel before they are to be distributed or after they are collected). High turnover of airline management personnel and flight attendants who are asked to voluntarily assist OTTI also complicates administration of this survey.
The survey, instructions, and a card requesting information from the flight crew are sent each month for selected survey flights to the Airline Survey Manager. The surveys are then put on board the survey flight. The information requested asks the flight crew to provide OTTI with a count of how many surveys were actually administered for that particular flight. For those flights returning this information, the number of completed questionnaires is taken as a percent of the total number distributed. This provides the reported response rate.
It is felt this method of calculation more accurately reflects the true passenger response rate. In addition, OTTI will continue its efforts toward improving response rates. Efforts will also continue to enhance the collection and reporting procedures for the return rates of international air passengers. .
Another effort to improve response rates has been the use of the boarding area survey method for selected carriers. The combination of the two methods has helped OTTI. The Boarding area survey method was designed to encourage a higher response rate. In support of this survey, conducted in boarding areas in airports instead of in-flight on board carriers, is the tighter control of the distribution and collection of the survey instruments, and the dependability and follow through of field survey contractors instead of flight attendants. The timing of returning the surveys to the contractor is also reduced.
More carriers are apt to participate in the program since the burden on the carrier is reduced. The addition of carriers would greatly enhance the coverage on all overseas and Mexican routes. Interference with airline boarding or check‑in procedures would be virtually non‑existent since all survey activity would cease at the first boarding call. The sample design would also be improved because of the even distribution of carriers departing from major airports and final distributions in foreign ports.
The problems associated with the boarding area surveys are: late arrivals to the boarding area
do not receive an opportunity to complete the questionnaire; passengers on board intermediate flights are not always allowed to disembark; passengers from feeder flights are not commonly available in the boarding area; many passengers and late arrivals are either not checked in early enough or are checked-in and wait in other parts of the airport (i.e., lounges, shops, VIP waiting rooms, etc.); the advent of common boarding lounges makes identification of passengers for the flight difficult.
In addition, there were international flights that continue to other U.S. ports before departing to a final overseas destination. Foreign flag carriers are prohibited from carrying passengers from one U.S. port to another if a final destination is overseas. However, U.S. flag carriers are not prohibited by this policy. Therefore, screening between domestic and international passengers is necessary. If the area surveying were limited to only final U.S. posts, those already on the airplanes could not be part of the response set. Alternatively, if intermediate international boarding areas were surveyed, the cost per survey is projected by the contractor to rise at least threefold. This is due to the time involved to screen extra personnel required for the extra flights.
Security clearances for surveyors must be obtained from the airport management and airline administrative offices. They, in turn notify station and gateway managers of the impending survey activity. With tighter airport security measures being implemented, OTTI must be assured by both airport and airline authorities that sub‑contractors will not encounter clearance prohibitions. Because every airport and every airline work differently procedurally, obtaining access is very time consuming and sometimes difficult. For example, in January through March 1991, OTTI could not survey any of the airlines using the boarding area method because of security reasons due to the Persian Gulf War.
With all of the problems associated with collection and returning the surveys, the current (a combined in‑flight and boarding area) methodology still is the most cost effective way to collect this information. OTTI spent time visiting the carriers which have proved very beneficial in helping us to improve the in-flight and boarding area survey methods. We have customized our survey procedures to meet the needs of the airlines. Where possible, we adopt our procedures to meet the carriers own internal survey methods. While this has had an added burden to the administration of the surveys to each airline, it has produced excellent results.
To improve response rates, OTTI has investigated the following ideas reported in the past clearance package as time, money and cooperation from the carriers dictated:
1. Better instructions on the flight packages. Improvements to the instruction sheets for the flight crew have been made. Several carriers assist us in translating the instructions into the language of the host country. We hope to work with others to continue this process.
2. Development of a monthly and quarterly airline response rate file. This was implemented in 1989, and continues to date. As you can see by the improved response rates, this tracking process has been one of the reasons for the improved response rates. It has also allowed us to find which carriers are not cooperating. These airlines have been approached at the specific ports where the problems occur as money has been available to do so.
3. Implement the enhanced airline utilization program. This project would explain to the airlines how these data can be used as a planning tool. Part of this system would look at problems of implementation and their affect on the quality of the data. Due to funding cuts we have not been able to implement this program.
4. Incentive programs for the Airline Service Managers (ASM) and Gateway Managers (GM). Certificates of appreciation have been provided to both groups. In addition, letters of thanks were sent to the CEO's for the U.S. flag carriers, and the Directors of North American operations for the foreign flag carriers. The head person for each airline was told if a certain GM had been very cooperative and instrumental in the success of the program for that airline and the ASM was recognized as well. We have learned, that the CEO and Directors, in certain cases, rewarded the ASM's and GMs. This program has proven to be successful when we have had the funds to implement.
5. Provision of pencils to survey passengers. It eliminates the excuse of not completing the survey because they do not have anything to write their responses to the questions asked.
4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.
The OTTI will continue with the existing in‑flight survey procedure, and continue to test the boarding area survey methodology. Both methods have their limitations and to use one method over another has not proven to be the most effective methodology. Even though the airline response rate has been shown to increase by administering a boarding area survey, the cost of this method is three times that of the SIAT. Thus, to implement the survey through all boarding area surveys is cost prohibitive. OTTI also investigated an onboard contractor methodology to collect the data. It too has proven cost prohibitive. Cost estimates to implement this type of program would be $1.4 million per year even assuming provision of free carriage.
1. Charting of the number of carriers that participate. This has been used since the initiation of the program, although better records are now maintained. Currently, we track a carriers’ performance over time, and now track it on an airport specific basis. This program will continue, and it will be used to help target which carriers are visited for the improved response rate program.
2. Better tracking of response rates. OTTI tracks monthly airline, and passenger response rates. The response rates have been used to identify poorly participating carriers. These carriers are called, sent letters, and visited to try and find out why we are not receiving the results we obtain from the other carriers. Steps have been taken to work with the problem airlines to customize our survey procedures to meet their needs. This change in philosophy is one of the main reasons, we have seen improved response rates, and new carriers joining the project.
3. Charting sales of the national reports. The lists are now reviewed closer to see who is buying the data. We look for repeat sales, and also look to determine who should be buying the reports. In fact, past subscribers were asked to help us improve the report formats. In addition, we are now working with several associations, and their members to further refine the data outputs.
4. Charting sales of the customized special reports and data tapes. This mechanism has helped us identify the heavy users, and possible new types of reports for the industry. This will continue in the future in greater detail. The special reports have also helped our contractor learn about the data. Many of the refinements to the data processing have been a result of the special reports generation. It has also helped us build up a large inventory of standard programming runs for future uses. Work has been limited in this area due to a lack of staff. Data is available to develop the analysis and when the next package is developed, we will show an analysis of the results.
5. Consider re-launching the investment program. This program received investment funds which allowed OTTI to increase the number of surveys distributed. Up till 2002, five of the seven investors stated they would again support this program. But all investors subsequently reduced their level of investment when compared to previous years. Collections totaled $50,000 which funded an additional 4,000 surveys for calendar year 2002. OTTI is also in the process of developing a government investment/user program. Our goal is to find other data users like BEA, DOT, INS, BLS, State, and Customs. These users will be charged for accessing the data base.
6. Investigate alternative methodologies. OTTI is constantly reviewing the research articles, and talking to companies about improving the program. To date, we have not found a better method than the one in current use, but are awaiting the results of the Request for Information (RFI). Major refinements have been made to this program. We hope to continue them with the help of our SIAT Airline and Industry Users Group. One idea is to combine two-three years of the sample and report it as the results for an individual year. This radical change would have to be tested. OTTI is looking into ways to have some universities help test this idea.
7. Airport Authority/Hotel & Motel/Rental Car/Credit Card Usage Reports and other new reports. OTTI hopes to work with the airport authorities, hotel/motels, credit card companies, and rental cars to get them to use SIAT data that is specific to the airport, hotel credit card use, or rental car firm. OTTI collects information on the ratings of airports, along with characteristics data on the airports arriving and departing passengers. OTTI also collects information on the hotel chains used by international travelers to and from the United States. It also has similar information on the use of rental car companies by company name. OTTI plans to develop and issue a federal register notice to release brand and ratings data. The industry will be used to determine how much and what levels of the brand data will be released. This value added information could help the industry better understand the linkages and potential partners they may have for marketing internationally.
8. Better Utilization of the I-94, I-92, and SIAT data proofing analyses. OTTI has expanded the analyses it completes to review the SIAT data each year to make improvements. We also compare the weighted sample to the known number of arrivals to the United States and their ports of entry. This test has shown the expansion process does work. It has also helped us find some of the small holes in data collection. If there are airlines, airports, countries, states, cities, sectors, or questions that indicate a problem, OTTI would like to use the data to confront and resolve the problem. Industry support through the users groups may also be enhanced after seeing the data and showing them why we are requesting their assistance.
9. Tracking the number of surveys used. By tracking usage by the different language versions, by respondents, and airlines, OTTI can improve inventory control, prepare for printing further in advance to save money on reprinting costs.
10. Track the response to particular questions. This is performed to help OTTI with future revisions. The questions with the poorest response will be reviewed to determine how they can be improved. It can also tell us if the survey is getting too long and needs to be scaled back. Watching the performance by question will help us obtain better response rates as we improve the wording, layout and design of future survey instruments.
All of these and other mechanisms will be used to help further enhance this very useful research tool for the travel industry.
5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
Judith Schwenk of the Transportation Systems Center, U.S. Department of Transportation (Volpe Center) was the original mathematical statistician who developed the survey sampling design and analysis procedures. The Volpe Center can be reached at (617) 494‑2488.
Dr. Reuben Cohen, Senior Vice President of Response Analysis was the statistician responsible for the technical direction of the program from April, 1984 through June, 1985. Response Analysis is now part of GFK Custom Research and can be reached at (609) 921‑3333.
Dr. Gordon Kubota, President of CIC Research, Inc., is the statistician that is responsible for the OTTI program beginning from July, 1985 and he may be reached at (858) 637-4000.
| File Type | application/msword |
| Last Modified By | richard champley |
| File Modified | 2008-10-30 |
| File Created | 2008-10-29 |