PPT Survey Supporting Statement Final rev (LMI 4-12-2013)

SUPPORTING STATEMENT FOR
PAPERWORK REDUCTION ACT SUBMISSION

Passport Demand Forecasting Study

SV-2012-0006

A. JUSTIFICATION

1. The United States border management community is tasked with the protection of the United States and its territories from foreign threats, the enforcement of immigration and customs laws, and the promotion of economic prosperity for the United States and our allies. The Department of State plays a vital role in this community through several offices, including its Bureau of Consular Affairs’ Passport Services Directorate. Passport Services is primarily responsible for issuing U.S. Passports to U.S. Citizens and Nationals who apply and meet all requirements. Passport Services assist U.S. Citizens and Nationals intending to travel internationally by issuing passports and protecting the integrity of the U.S. Passport as proof of U.S. Citizenship at home and around the world. Section 7209 of the Intelligence Reform and Terrorism Prevention Act (IRTPA), enacted on December 17, 2004, calls for the Secretary of Homeland Security (DHS), in consultation with the Secretary of State, to develop and implement a plan to require U.S. Citizens and Nationals to present a passport and/or other sufficient documentation of citizenship and identity when entering the U.S.

2. The objective of the statistical data and research sought is to estimate the overall demand for passport books and passport cards. Statistical data and information will aide Passport Services in developing demand projections monthly, semi-annually, annually and across multiple years as desired. The data is predicated and populated from ongoing monthly surveys. The surveys will include a nationally representative sample of U.S. citizens and U.S. Nationals age 16 and older. Passport Services will continue to use this data to monitor, assess, and forecast passport demand on a continuous basis for the U.S. population. The information gathered from the Passport Demand Forecasting Study Phase III will provide Passport Services the opportunity to refine volume and timing estimates on demand. The data gathered from the Passport Demand Forecasting Study provides Passport Services with a body of continuously updated and reliable statistics that is used for the purposes of staffing and resource allocation and budget planning for the coming years.

2. In concurrence with the Department of State’s endeavor to provide an electronic option for data collections, all monthly surveys will be conducted using surveys via mail that will have an identical internet/web version containing the same questions for those individuals who prefer to respond via the internet.

2. This collection will not duplicate any other information collection. The Department of State has attempted to establish an accurate demand figure by an extensive review of all known passport statistical databases. Passport Services has undertaken several prior surveys, with the initial Passport Demand Forecasting Study being implemented in 2007. While the previous studies were useful at that time for Passport Services early planning needs, the results are now dated and have been proven to be unreliable estimates for current and future demand.

2. The collection of information does not impact small businesses or other small entities.

2. The requested survey data is necessary in order to establish an accurate estimate of the volume and timing of passport applications for the purpose of monitoring, assessing, and forecasting passport demand on a continuous basis for the U.S. population. Without such information, Passport Services will not be able to make duly informed decisions on hiring and training staff, or increasing other resources and infrastructure to handle passport demand. If the survey is not conducted, Passport Services will not have reliable information to make resource-related decisions. This situation could result in the under or over estimation of demand, delayed passport issuance, adverse customer service, and excessive resource allocation, which would waste taxpayer money and Department resources.

2. This data collection consists of a voluntary survey. Respondents will be told at the onset of the survey that the data will be kept strictly private to the extent permitted by law and that identifying information will not be released outside of the Department. This notice will also be repeated to the respondents prior to soliciting any demographic information.

2. Passport Services will publish a 60-day Notice in the Federal Register requesting public comments.

2. Passport Services will not be providing any payments or gifts to respondents.

2. Respondents will be told at the onset of the survey that the data will be kept strictly private to the extent permitted by law and that identifying information will not be released outside of the Department. This notice will also be repeated to the respondents prior to soliciting any demographic information.

11. The collection of information does not ask questions of a sensitive nature.

11. The annual burden time for Passport Demand Forecasting Monthly Survey is estimated to be 8,000 hours. Each survey is estimated to take 10 minutes and will be conducted on a monthly basis. 4,000 respondents will be surveyed per month, for an annual total of 8,000 hours.

11. There is no cost burden to respondents associated with this collection.

11. The estimated cost to the Federal Government is $1,999,518.77 per year for the Passport Demand survey. A breakdown of the estimated cost can be found below.

CLIN#	Description	Qty	Unit	Unit Price	Total Cost
0001 FPP	Project Leader 2 ON	520	HR	$204.69	$106,438.60
0002 FPP	Senior Specialist 3 ON	1600	HR	$173.28	$277,248.00
0003 FPP	Analyst 1 ON	300	HR	$118.31	$35,493.00
0004 FPP	Analyst 2 ON	1040	HR	$103.72	$107,868.80
0005 FPP	Specialist 3 OFF	40	HR	$99.95	$3,982.00
0006 FPP	Research Specialist ON	720	HR	$79.59	$57,304.80
0007 FPP	Project Leader 2 ON	124	HR	$212.37	$26,333.88
0008 FPP	Senior Specialist 3 ON	320	HR	$179.76	$57,529.60
0009 FPP	Analyst 1 ON	100	HR	$122.75	$12,275.00
0010 FPP	Analyst 2 ON	240	HR	$107.61	$25,826.40
0011 FPP	Research Specialist ON	240	HR	$82.57	$19,816.80
0012 FPP	Other Services (Subcontractor, Sampling, Data Collection, Reporting)	1	LT		$1,269,401.69
	TOTAL				$1,999,518.77

11. The changes in burden are due to this collection being submitted as a reinstatement.

11. The information collected will not be published and is for internal statistical use only.

11. The Department of State will display the expiration date for OMB approval on the survey collection instruments.

11. Passport Services is not requesting any exceptions to the certification statement identified.

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

Overall Study strategy

More than ever before, key decision makers rely on empirical guidance for improving the operations of their organizations.  Such guidance, however, cannot be based on a plethora of raw data. Successful management requires actionable intelligence, vital to which is reliable information that has to be distilled from representative data. As such, scientific sample design and diligent survey administration; effective analysis of survey data; and coherent interpretation of the results are critical steps towards the ultimate goal of this forecast study. That is, providing the Bureau of Consular Affairs (CA/PPT) with reliable projections of demand for various passport products for efficient resource allocations and staffing. Every step of our proposed strategy for this study is guided by this roadmap and is cognizant of the fundamental role survey data play along this path.

Another fundamental tenet of our strategy is recognition of the fact that rigorous survey research programs are designed and implemented to minimize the so-called Total Survey Error, as depicted in the following diagram. Under this comprehensive framework each component of error receives proper attention, since with imbalanced focus on any particular source of error other error components will gain opportunities to grow and create weak links in the survey process.

Figure 1. Components of total survey error

Total Survey Error

Errors of Non-observation

Errors of Observation

Errors of Processing

Errors of Dissemination

Sample Coverage

Response Rates

Instrument

Data Collection

Data Editing & Compilation

Imputation & Weighting

Analysis of Survey Data

Interpretation & Action Plans

Striking an optimal balance here requires academic knowledge, hand-on experience, and transparent execution. Conducive to this objective is the solid – yet intuitive – design the Passport Services has envisioned for this of this study, the main components of which are outlined in this section. Specifically, in what follows we will discuss:

1. Design and selection of representative samples that are probability-based;

2. Collection of reliable data using a respondent-friendly protocol;

3. Effective data enhancements procedures; and

4. Reliable demand projections and timely reporting of the results.

1. Design and Selection of Representative Samples

This forecast study will employ an address-based sampling (ABS) methodology to reach a probability-based sample of the US households every month. Increasingly, survey researchers are considering address-based sampling (ABS) methodologies to reach the general public for data collection applications. Essentially, there are three main factors for this change:

• Evolving coverage problems associated with the traditional methods of sampling;

• Eroding rates of response to single modes of contact; and on the other hand

• Recent improvements in the databases of household addresses available to researchers.

Indeed, recent advances in databases of household addresses have provided a promising alternative for surveys that require contacts with representative samples of households. The Computerized Delivery Sequence File (CDSF) of the USPS is a database that contains all delivery points in the US, a summary of which is provided in the following table. For this survey the monthly sample of addresses will be obtained from the enhanced ABS frame developed by Marketing Systems Group (MSG).

Table 1. Distribution of the CDSF delivery type points

Delivery Point Type	Count
City Style/Rural Routes	115,944,396
Traditional P.O. Box	14,239,146
Only Way of Getting Mail (OWGM) P.O. Box	1,404,611
Seasonal	858,184
Educational	94,678
Vacant	3,630,143
Throwback	275,670
Drop Points	731,616
Augmented addresses (by MSG)	150,587
Total	137,329,031

By identifying the latitude and longitude of each address MSG is able to create a one-to-one correspondence between the Postal geographic indicators, which are suitable for mail delivery, and those suitable for sampling designs based on the Census geographic definitions. Subsequently, the resulting database is augmented with a list of geo-demographic indicators to evolve the raw CDSF into an effective sampling frame suitable for selection of probability-based samples.

For this study a monthly sample of approximately 25,000 addresses will be selected to represent the nation. This sample will be selected from the MSG-enhanced ABS frame, which will be updated on a quarterly basis. Each sample address will be name and telephone matched, with the expectation that over 55 percent of addresses will link to a landline telephone number. Moreover, each address will be appended a series of geo-demographic data by relying on public and commercial sources.

As detailed later, matched telephone numbers will be used as part of our non-response follow-up to maximize response rates, whereas the appended ancillary data will support our non-response bias analysis to help develop effective weighting procedures. While a subset of such data will be available for individual addresses, others will be retrieved at higher levels of aggregation, such as Census Block Group (CBG) or ZIP Code.

On quarterly basis, 25 regional estimates will be required for this study, corresponding to the CA/PPT’s 25 geographically-based agencies where U.S. citizens may apply in person for a passport. To ensure that survey estimates for each of the agencies will be of equal precision, each monthly sample will be stratified accordingly to include an equal number of sample addresses per agency. Specifically, the total monthly sample of approximately 25,000 addresses will be stratified into 25 strata of 1,000 addresses each. From the monthly sample of 1,000 addresses per agency, it is expected that about 160 completed surveys will be secured each month, which when aggregated will result in about 480 = 160 × 3 completed surveys every quarter.

The table above provides the distribution of addresses and expected sample size. Note that less than 3 percent of all addresses are currently marked as “vacant” according to the CDSF. Given that a small percentage of such addresses can be indeed occupied by the time the survey administration begins for a given month, it is possible to include a nominal fraction of such households as part of the sampling frame to minimize under coverage.

2. Data Collection

It is anticipated that each month about 4,000 completed surveys will be secured using multiple methods of survey administration. For this study both web and telephone modes of data collection will be used to produce the highest rates of response by making the survey experience as convenient for respondents as possible. Briefly, our survey administration protocol consists of the following main steps:

• One week before the beginning of the month a cross-sectional random sample of approximately 25,000 addresses will be selected from the latest MSG-enhanced version of the CDSF.

• Five days before the beginning of the month, invitation letters will be mailed to sample households in Alaska and Hawaii.

• Four days before the beginning of the month, invitation letters will be mailed to sample households in the Pacific and Mountain time zones.

• Three days before the beginning of the month, invitation letters will be mailed to sample households in the Central and Eastern time zones; this staggered mailing is designed to allow for actual receipt of the invitations to occur simultaneously throughout all selected households nationwide.

• Sample households may begin responding to the invitation by web or inbound telephone call upon receipt of the letter.

• On the first day of the month, outbound telephone calls will begin to all non-responding households with telephone numbers appended to their records;

• On the ninth day of the month, reminder letters will be mailed to the remaining non-responding households in Alaska and Hawaii;

• On the tenth day of the month, reminder letters will be mailed to the remaining non-responding households in the Pacific and Mountain time zones;

• On the eleventh day of the month, reminder letters will be mailed to the remaining non-responding households in the Central and Eastern time zones;

• Data collection will close on or around the 25^th of each month upon completion of 4,000 total interviews within that month (to occur no later than the final day of the month); this total will include web responders, inbound telephone interviews, and outbound telephone interviews conducted to achieve the highest possible response rate.

Prior to the start of the month, the survey instrument will be programmed and tested in two versions: one for online telephone interviewing (whether inbound or outbound); and the other for self-administration via the web. Data will be collected throughout the month using all the available modes, using one single sample file containing both the records with phone number matched and the unmatched records. The sample file will be simultaneously managed regardless of the mode of survey completion. Survey responses, overall progress, progress by geographical areas, and sub-groups of interest can be monitored in real time as every mode is working into the same program and data file.

Between 10 days and 4 days prior to the start of the month, the sample will be matched to external records systems to generate as many mailings as possible addressed to the surname of the household members. The proposed salutation strategy plans to insert name when available, in addition to including “or Current Resident” on all outgoing mail pieces.

Within each responding household eligible for this study an adult householder familiar with the passport needs of the given household will be asked to participate on behalf of all members of his/her household. While web will be encouraged as the preferred mode when appropriate, inbound and outbound telephone will be used as alternative means of completing the survey to increase response rates. Both the invitation as well as outbound and in-bound phone efforts will stress the importance of selecting a respondent within the household who is at least 18 years of age and is familiar with the travel habits and travel document needs of all household members. Subsequently, the respondent will be asked to provide a complete enumeration of the entire household as well as responding to questions about the travel needs for both the selected respondent and all eligible household members.

Records in high-incidence Hispanic block groups (defined as blocks that are 75% Hispanic) and those matched with Hispanic surnames will be flagged for a bilingual (English/Spanish) invitation letter to be mailed to those addresses, while the remaining invitation letters will be English only. The web version of in the instrument will be available to all responders in their choice of English or Spanish. Bilingual interviewers will always be available to conduct inbound and outbound interviews in either English or Spanish, as preferred by the respondent.

All completed surveys will be managed through a unified system and sample file. This will ensure that if a respondent completes the survey online or by calling the toll-free number, he or she will not be contacted by an outbound dialing interviewer. Returns of invitations determined by the USPS to be Undeliverable as Addressed are expected to be received starting around the seventh day of the month. These records will be processed and the sample file updated accordingly. Original invitations that had been name-matched during the sample preparation phase and mailed addressed to that surname’s household but are returned as undeliverable by the USPS will be revised and addressed to “Current Resident” in the reminder invitation mailing.

General public survey interviewing is usually dialed in the evening between 6:00 p.m. and 9:00 p.m. local time and during weekends. Day time outbound calls are also conducted as dictated by sample needs and respondent preferences. Experienced daytime interviewers will be briefed to handle inbound calls and callbacks. The data collection team will keep its facilities open and available for inbound and outbound calls from 9:00 a.m. to 11:00 p.m. Monday through Friday; 11:00 a.m. to 6:00 p.m. on Saturday, and noon to 10:00 p.m. on Sunday, Eastern Standard time. In addition, a west coast facility will remain open to accommodate outbound dialing through 9:00 p.m. local respondent time in Alaska and Hawaii.

All interviewers used for data collection will be experienced survey research interviewers with prior training in use of CATI. The training of STR interviewers surpasses industry standards, averaging about 24 hours before starting on any study. Their training includes an overview of survey research, equipment, and quality standards, as well as role playing and internal survey work. In addition, STR has an interviewer mentoring program whereby new interviewers are partnered with experienced supervisors and executive interviewers to allow for one-on-one training. Interviews are digitally recorded and reviewed with interviewers as part of the process of becoming high-quality, dedicated interviewers.

• Degree of accuracy needed for the purpose described in the justification

As mentioned earlier, Passport Services requires quarterly estimates for each of its 25 geographically-based agencies with a margin of error no larger than ±5% at the 95% confidence level in addition to monthly national level estimates with a margin of error no larger than +2%at the 95% confidence level. In order to meet these requirements, the LMI Team will complete about 4,000 surveys per month. The sampling error associated on average for the 25 quarterly agency estimates based on a sample size of approximately 480, including anticipated design effect, is about ±5% at the 95% confidence level. The sampling error associated with the national estimates based on a sample size of 4,000, including anticipated larger design effect, is about +2% at the 95% confidence level. .

• Unusual problems requiring specialized sampling procedures

Passport Services desires that the survey be representative of all U.S. citizens and U.S. Nationals age 16 and older. At this time, there are no anticipated problems requiring specialized sampling procedures. The LMI Team can and will make adjustments as necessary.

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden

Passport Services has stated earlier that their need to project passport products demand requires monthly estimates and adjustments. Therefore, we have the LMI Team conducting monthly surveys. In addition, we have asked the LMI Team to minimize respondent burden by limiting the number of non-passport forecast questions. These non-forecast specific questions all feed into a better overall understanding of passport product demand. Passport Services has a need for feedback on passport customer service performance, determining reasons behind households/household members deciding to acquire passports, households’/household members’ expectations of future international travel, and potential interest in new passport products. The LMI Team will implement a rotating schedule of four modules (the four enumerated above) of four to ten questions each so that Passport Services gets the information it needs to complement its forecast data, but respondents will only be exposed to an additional five questions a month. Since the monthly surveys are each nationally representative, the modular approach retains estimation accuracy while improving CA/PPT’s ability to acquire complementary data to the forecast data.

3. Methods to Maximize Response Rates and Adjust for Non-Response

All practical steps will be taken to maximize response rates to this survey. A sample of such steps will include:

• Building Credibility Through Use of an Advance Mailing:

• Constructing the survey instrument to be as respondent-friendly as possible

• Sponsor identification via signed letters by a Department of State official

• Use of Department of State Graphics and color scheme

• Reference to Web site URL’s for study legitimacy validation

• Provision of toll-free information line to field respondents’ questions

• Call to action to respond by a specific deadline

• Notification of on-going efforts to be conducted through the month to reach the selected household

• Outbound Calling Effort:

• Up to ten attempts rotated through various day times: (early evening, later evening, and weekends)

• Day time calls as requested by respondents

• Specific call back scheduling

• Offering respondents the most convenient mode of data collection

• Reminder Attempts:

• USPS returned mail will have delivery addressed to “Current Resident” for the reminder mailing

• Reminder effort will reiterate the importance of the study and contain all information in the original mailing in terms of rapport building

• Establishment of legitimacy and availability of web and phone information for respondents who may have questions

The data will be adjusted to deal with issues of non-response. We describe our approach to survey non-response adjustments and enhancements below.

Data from all scientific surveys are weighted before the resulting data can be used to produce reliable estimates of population parameters. While reflecting the selection probabilities of sample units, weighting also attempts to compensate for practical limitations of a sample survey, such as differential non-response and under coverage. Furthermore, by taking advantage of auxiliary information about the survey population, weighting can reduce the bias of survey estimates by enabling the responding subset of the sample to better represent its target universe. This is of particular importance for a tracking survey of this nature, since some of the month-to-month random variations can be minimized by weighting the data before survey estimates are produced.

Typically, the weighting process entails three major steps. The first step consists of computation of design weights as the reciprocal of selection probabilities. In the second step, design weights will be adjusted for non-response – a process that will be guided by a comprehensive non-response bias analysis. In the third step, non-response-adjusted weights will be further adjusted to known population estimates to compensate for sampling frame inadequacies. All along, weighting adjustment steps will go through a series of quality control checks to detect extreme outliers and to prevent computational inefficiencies.

For this survey we will use the WTADJUST procedure of SUDAAN to weight the monthly survey data¹. Unlike traditional raking procedures that are based on iterative proportional fitting, this model-based approach can incorporate more main effects and lower-order interactions of variables when computing weights. Moreover, because this procedure allows limiting the resulting weight adjustment factors, one can eliminate extreme weights early in the process and control the variability of the final weights. Consequently, with this alternative it is generally possible to achieve balance with respect to an expanded set of control totals while at the same time reducing the variance of weighted statistics.

Our non-response bias analysis will include comprehensive comparisons of the geo-demographic composition of respondents with their corresponding estimates as reported by Current Population Survey (CPS), or the American Community Survey (ACS) for smaller geographic levels. Each month a fresh round of non-response bias analysis will be carried out to provide current guidelines for weight adjustments. In addition to compensating for differential non-response patterns based on the data collection mode, this approach will also take into account seasonal variations when creating the final weights.

It should be noted that prior to non-response bias analysis and computation of survey weights, it will be necessary to impute missing data that will result from item non-response and data items that fail edit checks. Since missing data can create inefficiencies for demand projections, when appropriate we will use the method of weighted sequential hot-deck² to impute missing survey data. By incorporating the sampling weights, this method of imputation will reflect the unequal probabilities of selection in the monthly sample while controlling the expected number of times a particular respondent’s answer will be used as a donor to replace missing values.

Finally, survey estimates can only be interpreted properly in light of their associated sampling errors. Since weighting often increases variances of estimates, use of standard variance calculation formulae with weighted data can result in misleading statistical inferences. For this survey we will use SUDAAN and SAS to compute weighted demand, variances of which will be approximated using the Taylor Series Linearization technique. Without this any projections of demand will be subject to confidence intervals with artificially narrow widths. That is, survey estimates will show a larger level of confidence than what they actually should.

4. Testing

The survey questionnaire will be internally pre-tested for timing, content, and clarity. Moreover, STR will engage in a formal initial pre-test of the survey instrument, to be conducted with a sample of 30 households to confirm that the screening questions and procedures, as well as all survey logic are working as intended. Although the pre-test will be designed as a confirmatory procedure, if any issues are uncovered with survey instructions (such as item wording or incomplete response categories) revisions will be proposed and incorporated into the final survey materials upon receipt of agency approval.

Survey procedures will be tested in several ways and will be on-going when any changes are made to the base survey instrument. These tests will be involving no more than nine participants to examine the comprehensibility, structure and order of survey questions.

• Moreover, as part of our testing procedures different survey administration options can be examined to improve response rates. While minor in nature, these options may include: Addressee Mailing Information

• Named salutations when addresses are name-matched vs. unmade salutation using:

• Current Resident

• State Resident

• Other salutations

• Initial and Reminder Mailing Treatments

• Envelope Tests for Open Rate

• Letter Tests for Click Through and Response Rates

• Call to action and delivery deadline

• Web vs. in-bound phone information placement

• Ordering of other invitation information

• Logos and stationary options

• Timing of Mailing

• Testing Efficacy of Pre-notification Calls

• Testing Efficacy of Reminder Calls

All testing would be done in the conduct of the monthly survey utilizing replicated sub-samples of the main sample file.

Based on its experience, STR has developed mailing materials and protocols which should produce the rates of response required to reach the desired number of monthly interviews.  At the current time, STR plans no testing /experiments of different mailing procedures as a matter of course during the conduct of the fieldwork.  That said, if mailing efforts fail to produce the desired number of interviews, different survey administration options can be examined to improve response rates.  Any contemplated mailing procedure changes will be based on observed response rates, mail return rates, and respondent feedback where available.  Any proposed mailing procedure changes or experiments will be submitted for approval prior to implementation.   While minor in nature, these options may include:

• Addressee Mailing Information

• Named salutations when addresses are name-matched vs. unmade salutation using:

•  Current Resident

• State Resident

• Other salutations

• Initial  and Reminder Mailing Treatments

• Envelope Tests for Open Rate

• Letter Tests for Click Through and Response Rates

• Call to action and delivery deadline

• Web vs. in-bound phone information placement

• Ordering of other invitation information

• Logos and stationary options

• Timing of Mailing

• Testing Efficacy of Pre-notification Calls

• Testing Efficacy of Reminder Calls

All testing would be done in the conduct of the monthly survey utilizing replicated sub-samples of the main sample file.

5. Contact Information for Statistical ConsuLTants and Analysts

Mansour Fahimi, Ph.D., (610) 994-8374. Marketing Systems Group – Sampling Design and Analysis

Gregg Kennedy, (215) 870-8656, Survey Technology and Research Center – Design and Collection

David Levin, Ph.D., (571) 633-7925, LMI – Sampling and Survey Design and Analysis

1.R.E. Folsom and A.C. Singh (2000). “The Generalized Exponential Model for Sampling Weight Calibration.” Proceedings of the Section on Survey Research Methods of the American Statistical Association, pp. 598-603.

2.Iannacchione, V.G. (1982). Weighted Sequential Hot Deck Imputation Macros. In Proceedings of the Seventh Annual SAS User’s Group International Conference (pp.759–763). Cary, NC: SAS Institute, Inc.