Survey Methodology Agreement

OMB No.1405-XXXX 2/4/2021

SV-2007-0021

U.S. Passport Demand Study Phase II

Proposal for Revision of Survey Methodology

OMB No.1405-XXXX
SV-2007-0021

This is to acknowledge that on July 11, 2007 Gallup staff discussed with OMB officials a number of enhancements for the quick-response passport and travel survey¹ designed to increase the response rate and meet the methodological requirements that OMB has established for surveys. The discussion also covered the use of a more robust non-response bias study. Based on these discussions, Gallup subsequently briefed the Passport Office of OMB’s requirements for increasing the survey response rate and conducting a non-response study, as well as the additional resources and time that would be required to implement all of these measures. Based on our discussions, the Passport Office requested that Gallup formulate a workable solution that would meet OMB’s core requirements while minimally impacting the delivery schedule.

Along these lines, we believe that of the three survey enhancements discussed with OMB—(1) a pre-notification letter to be sent via first-class mail;² (2) expanding the call design to 7x7; (3) a follow-up letter to be sent to “refusal” and “non-contact” households via first class mail—that the most vital enhancement involves expanding the call design from the current 5x5 to a 7x7 design. The expanded call design will allow up to 7 calls to make contact with the sampled household and up to 7 additional calls to identify the target respondent. Even though additional calendar time will be required to work through the calling algorithms for this design, our experience suggests that it can be accomplished by adding about two weeks to the current data collection schedule. While the pre-notification and follow-up letters are important enhancements in their own right, our experience suggests that their impact will be limited owing to the fact that a “good” mailing address will be available for only a small proportion of the sample, anywhere from 35% to 40%, of all sampled telephone numbers. Consequently, we believe that the greatest impact is to be derived from the expanded callback design and, as we describe below, a more robust non-response bias study. We anticipate the expanded call design increasing the data collection schedule from 6 to 8 weeks and increasing the targeted response rate from 35% to 45% or more. Gallup is prepared to implement the expanded call design with OMB and agency approval.

Non-response Bias Study: Given that we anticipate achieving a 45% response rate for this study, we propose to conduct a more robust non-response bias study in accordance with OMB requirements. In a recent work, Groves³ reports that there is no consistent relationship between response rates and non-response bias. As such, a lower response rate may not necessarily cause or result in non-response bias. However, it does not imply that this finding will be true in all circumstances. We, therefore, plan to undertake a separate non-response bias study to examine the non-response patterns and to assess the potential for non-response bias in this survey once the primary data collection for it has been completed.

The pool of non-respondents for this study will be stratified into two groups: (i) Non-contacts and (ii) Refusals. Within each stratum, the goal will be to complete about 100 interviews in each group. Based on an anticipated response rate of around 7%, we plan to draw a total sample of approximately 3,000 for the non-response study. The sample size will be allocated proportionately across the two strata. Although it is difficult to predict the relative size of these two strata, we anticipate the non-contacts will account for about 60-70% of all non-respondents. The sample, within each stratum, will be allocated proportionately across these two groups.

The mode of data collection for the non-response study will be telephone using a 7x7 call design, with up to 7 calls devoted to contacting the household and up to 7 calls to interview the targeted respondent. All sample members will be offered a cash incentive of $20 to complete the telephone interview. A check will be mailed to the respondent’s household following completion of the survey along with a letter thanking them for their participation. The letter will be sent on Gallup letterhead under the signature of the Project Director. We anticipate a data collection period of about 4 to 6 weeks to complete the call design and achieve the targeted response rate.

The non-response bias analysis will involve examining (i) the representativeness of the sample of respondents for the target population and (ii) the difference between the survey estimates generated for the respondents and the non-respondents. In assessing representativeness, we will conduct an analysis of demographic characteristics by geographical area using census or similar external data sources. In comparing the survey estimates, we will use the weighted estimates for the two groups (respondents from the main study and non-respondents from the non-response bias study). If necessary, we may, for the purpose of better understanding the non-response patterns, investigate the suitably of examining select subgroups within the group of respondents for the main study. We may, for example, examine: (i) those that are "easiest to reach and interview" as measured by records of calls in our CATI system; (ii) those that are "more difficult to reach" (require more callbacks) compared to group (i) above. The goal of the non-response bias study will be to detect if significant and policy-relevant differences exist between the survey estimates for respondents and the non-respondents. If such differences are found, we will explore the necessity of making the necessary adjustments to the survey estimates to minimize the non-response bias in the main study findings.

A primary goal of this study is to obtain data on land border crossers by the beginning of September, as desired by key State Department and officials. Following the receipt of emergency approval for this survey, however, the Department intends to apply for a three-year approval (through the standard OMB process) for ongoing study of passport demand. The Department will strive to incorporate OMB’s recommended methodology in these ongoing efforts.

1 Our target population for this study is defined as land border crossers, that is, those individuals who crossed a land border in the past 12 months or plan to cross a land border in the next 12 months. A fixed reference period of 24 months will be used to determine eligibility for the survey.

2 The fourth enhancement involved an automated search to obtain mailing addresses for all sampled telephone numbers for the pre-notification and follow-up mailings.

3 Groves, Robert M. 2006. “Nonresponse rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70: 646-675.

2