Gallup OMB LBC Supporting statement PART B 8-12-08

Supporting Statement for Paperwork Reduction Act Submission

U.S. Passport Demand Study Phase II

OMB No. 1405-0177, SV-2007-0021

Part B

B. Collections of Information Employing Statistical Methods

1. The objective of this survey research is to estimate the overall demand for applications for passports and passport cards. For the Land Border Crossing Survey, the objective is to estimate the number of U.S. citizens who travel to Canada and Mexico by land and who plan to apply for a passport as a result of the change in U.S. law and regulations requiring a passport for entry back into the U.S. Passport demand will be estimated based on a telephone survey of a representative sample of U.S. citizens in households, which will be selected principally from areas with the largest numbers of existing and prospective future land border crossers. The potential respondent universe will consist of all U.S. citizens who live in a household and who have crossed back into the U.S. at the Canadian or Mexican border by land in the previous 12 months or plan to cross the border in the next 12 months. Within each such household, one adult U.S. citizen will be selected randomly to complete the survey. The 2007 LBC survey results indicated that approximately 33 million U.S. citizens cross a U.S. land border each year.

Statistical procedures will be used for the focus group selection, the Land Border Crossing Survey, and Non-Responsive Bias Survey sampling and respondent selection. A focus group will be convened once per quarter. The proposed Land Border Crossing study will screen for households containing at least one person who is a U.S. citizen and has crossed the border in the previous twelve months or plans to cross the border in the next twelve months. As described above, we expect about 33 million U.S. citizens to cross the border in a 24-month period. Other U.S. citizens who do not meet these criteria may also plan to apply for passports, but these individuals are not considered within the scope of this survey as they are unlikely to be able to accurately forecast their future travel behavior. The estimate of the demand for passports will be based solely on demands from U.S. citizens with past experience or near-future intentions of making land border crossings.

2. The following procedures will be employed for the collection of information.

To implement this LBC design, all households in the U.S. will be stratified into two sampling strata: a high-density stratum and a low-density stratum, where density refers to the proportion of households that contain one or more persons that have crossed the Canadian and/or Mexican borders at a land POE in the last 12 months or plans to do so in the next 12 months. The two strata will be formed as follows: Around each of the POE, we will identify geographic areas (states, cities, counties) that are likely to house relatively large numbers of land border crossers. We expect that most of these areas will fall within a 200-mile radius of a large land POE. The high density stratum will be formed around all POE along each border. In many locations, smaller POE located near high-volume POE will automatically be included within the areas defined by proximity to the larger POE. To improve sampling efficiency, we will evaluate whether the smallest POE along each border should be allocated to the low-density stratum, because the numbers of crossings and the populations of the surrounding areas may be too small to influence the survey results. We anticipate that focusing on the eight to 10 largest volume POE on each border to generate the high-density stratum will permit us to capture the residences of at least 90 percent of the land border crossers along both borders. Once the high-density stratum is established around the high-volume POE, the remainder of the country will be assigned to the low-density stratum. The target areas will be defined in terms of states/cities/counties or in terms of entities that can be mapped accurately to area codes/exchanges (typically county boundaries). The size, in terms of number of households, of each stratum will be known only after they are constructed. However, it is expected that the high-density stratum will include not more than approximately 30 million households and the low-density stratum will include about 95 million households.

For the purpose of sampling, four domains will be generated as shown below. A total of about 4,000 completed interviews divided across four analytical domains will be completed.

Eastern Western

Segment Segment Total

Canada 1,000 1,000 2,000

Mexico 1,000 1,000 2,000

Total 2,000 2,000 4,000

The Eastern and Western segments will be defined using pragmatic divisions of the land crossing volumes along each of the two borders. As a result, the final distribution of completed interviews may somewhat differ from the equal allocations shown above.

For each of the four domains (Canada/Mexico and Eastern/Western segments), the geographic areas will be stratified into high and low density strata and sufficient sample size will be allocated to each of these areas to produce the required number of completed interviews. We will select RDD samples independently for each stratum. The sample will be obtained using a list assisted telephone sampling design. Assuming a statistical design effect of about 1.25, the number of interviews as presented in the table above will ensure a precision of about + 3.5 percentage points, which creates an estimation of a proportion around 50% for each of the analytical domains. However, the exact value of the design effect could be higher, thereby reducing the effective sample size. For national estimates based on 4,000 completed interviews, the margin of error is not likely to exceed +3 percentage points and is more likely to be less than + 2 percentage points.

As allocated, the yield of interviews with land border crossers in the low-density stratum will be very small compared to that for the high-density stratum. If the incidence rate for this stratum is extremely low (e.g., less than 5 percent), it may be statistically and operationally optimal to restrict data collection in the low-density stratum and redirect resources to enhancing the response rates and data quality for the high-density stratum.

Data will be collected using a telephone survey of approximately 10 minutes duration. A “seven-plus-seven” call design (up to seven calls to establish contacts with an eligible adult at the sampled household and up to seven more calls to complete the interview) will be used to complete about 4,000 interviews over a period of about 6 to 8 weeks.

3. The task of completing about 4,000 surveys for this study must be done in a relatively short period of time (6 to 8 weeks). As a result, it may not be feasible to apply some of the methods (such as advance letters, rigorous refusal conversion techniques etc.) that are known to have a positive impact on the response rates. However, within the time constraint, all possible steps will be taken to help increase the response rate for this study.

All supervisors and interviewers assigned to this study will be chosen using a highly discriminating selection interview that favors strong communications skills. Moreover, all interviewers assigned to this study will have had recent prior experience conducting CATI surveys of the general public on important issues. This approach will avoid the cost of providing general interviewer training and will permit focusing training strictly on the questionnaire and sampling procedures for this study.

In addition to the impact of training protocols on promoting quality in data collection, a systematic sample of each interviewer’s work will be monitored in real time using unobtrusive technology that permits the monitor both to hear the interview transaction and to view the interviewer’s keystrokes as responses are recorded. The sampling rate for monitoring interviews starts high during the first week of calls and is reduced during the field period (but never stopped) so that a total of 5 to 10 percent of each interviewer’s work is monitored across the entire survey.

After each monitoring session, supervisors will review the interviewer’s performance by pointing out any deficiencies, providing specific guidance to ensure that quality goals are met or exceeded, as well as recognizing and praising the excellent performance of interviewers. Supervisors will hold debriefing sessions with small groups of the interviewing staff concerning the progress of the data collection effort. These sessions are designed to identify any systematic problems with authorized data collection procedures that may be impeding the attainment of the study’s goals, to develop effective solutions to such problems, and to fine tune survey procedures to ensure high levels of respondent cooperation while also promoting high efficiency in all aspects of the field effort.

Efforts will also be made to convert refusals to the extent possible and thereby reduce the effect of non-response. While “hard” refusals (those who swear at the interviewer, ask to be taken off our list, or threaten the interviewer in any way) will not be recontacted, those who were “soft” refusals (indicating they are not interested, don’t do surveys, or prefer not to participate) will receive additional calls for completing the survey.

Gallup will also perform a Non-response Bias Study. Given that Gallup anticipates achieving a 45% response rate for this study, it will conduct a more robust non-response bias study in accordance with OMB requirements. In a recent work, Groves¹ reports that there is no consistent relationship between response rates and non-response bias. As such, a lower response rate may not necessarily cause or result in non-response bias. However, it does not imply that this finding will be true in all circumstances. Gallup, therefore, will undertake a separate non-response bias study to examine the non-response patterns and to assess the potential for non-response bias in this survey once the primary data collection for it has been completed.

As a routine procedure, Gallup monitors non-response during the implementation of the survey data collection. During this time, every effort is made to ensure that the data collection coverage is comprehensive across all key subgroups of the sample in order to minimize uneven non-response among the subgroups and to achieve the overall response rate target. In the event that Gallup detects a significant non-response bias, it’s statisticians and methodologists will provide a detailed plan for adjusting the survey results of the main study. The adjustments may include revising the weighting schema to carry out specific non-response adjustments.

Regular non-response bias studies will be conducted immediately following the Land Border Crossing (LBC) surveys to establish an on-going process for monitoring non-response bias. Based on the findings, CA will implement necessary changes to future surveys on a continuing basis. Necessary steps like refusal conversions will be taken to ensure a minimum refusal rate to start with. If certain groups (based on age, gender or geographic location for example) are underrepresented in the survey, CA may consider oversampling some of these groups to the extent possible in future surveys. Finally, the weighting procedure may be modified to adjust for specific non-response by including appropriate variables in non-response/post-stratification weighting scheme.

The pool of non-respondents for this study will be stratified into two groups: (i) Non-contacts and (ii) Refusals. Within each stratum, the goal will be to complete about 100 interviews in each group. Based on an anticipated response rate of around 7%, we plan to draw a total sample of approximately 3,000 for the non-response study. The sample size will be allocated proportionately across the two strata. Although it is difficult to predict the relative size of these two strata, we anticipate the non-contacts will account for about 60-70% of all non-respondents. The sample, within each stratum, will be allocated proportionately across these two groups.

The mode of data collection for the non-response study will be telephone using a 7x7 call design, with up to 7 calls devoted to contacting the household and up to 7 calls to interview the targeted respondent. All sample members will be offered a cash incentive of $20 to complete the telephone interview. A check will be mailed to the respondent’s household following completion of the survey along with a letter thanking them for their participation. The letter will be sent on Gallup letterhead under the signature of the Project Director. We anticipate a data collection period of about 4 to 6 weeks to complete the call design and achieve the targeted response rate.

The non-response bias analysis will involve examining (i) the representativeness of the sample of respondents for the target population and (ii) the difference between the survey estimates generated for the respondents and the non-respondents. In assessing representativeness, we will conduct an analysis of demographic characteristics by geographical area using census or similar external data sources. In comparing the survey estimates, we will use the weighted estimates for the two groups (respondents from the main study and non-respondents from the non-response bias study). If necessary, we may investigate the suitability of examining select subgroups within the group of respondents for the main study, for the purpose of better understanding the non-response patterns. We may, for example, examine: (i) those who are "easiest to reach and interview" as measured by records of calls in our CATI system; (ii) those who are "more difficult to reach" (require more callbacks) compared to group (i) above. The goal of the non-response bias study will be to detect whether significant and policy-relevant differences exist between the survey estimates for respondents and the non-respondents. If such differences are found, we will explore the necessity of making adjustments to the survey estimates to minimize the non-response bias in the main study findings.

4. Questionnaires and procedures will be tested in several ways. The questionnaire will be internally pre-tested by DoS and contractor personnel for timing, content and clarity. Although the questionnaire is very brief and not expected to present difficulties, a review using cognitive laboratory testing methods (viewing the respondent as he or she answers questions, follow-up probing questions to ensure understanding as intended of the questions, etc.) involving fewer than ten participants will be undertaken to examine the comprehensibility, structure and order of survey items.

Gallup will also engage in a formal pre-test of the survey instrument, to be conducted with a sample of 50 households in the high-density stratum (25 on each border) to confirm that the screening questions and procedures, as well as all items in the main survey questionnaire are working as intended. Although the pre-test will be designed as a confirmatory procedure, if any issues are uncovered with survey instructions, item wording, or response categories during the process, revisions will be proposed and incorporated into the final survey materials upon receipt of agency approval. In the event that a formal pre-test is not possible given the time constraints, Gallup will conduct the standard 9 pre-test interviews to assess the operation of the questionnaire.

5. The Gallup Organization developed the survey design and will be responsible for collecting, processing and analyzing the data and presenting findings to CA/PPT. The following individuals were consulted in developing the survey design, the sampling plan, and statistical aspect of the study.

Dr. Manas Chattopadhyay Dr. Cheoleon Lee

Chief Statistician Sr. Statistical Analyst

The Gallup Organization The Gallup Organization

901 F Street, NW 901 F Street, NW

Washington, D.C. 20004 Washington, D.C. 20004

202-715-3179 202-715-3127

Dr. Sameer Abraham Calvin Jones, ABD

Partner & Managing Research Director Sr. Statistical & Research Consultant

The Gallup Organization Statistical and Evaluation Research

Government Group 24190 Lenah Woods Place

901 F Street, NW Aldie, VA 20105

Washington, D.C. 20004 703-957-4110

202-715-3180

Dawn Royal, M.A.

Engagement Manager

The Gallup Organization

901 F Street, NW

Washington, D.C.20004

248-936-4152

1 Groves, Robert M. 2006. “Nonresponse rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70: 646-675.

4