Supporting_Statement_B

INFORMATION COLLECTION

SUPPORTING JUSTIFICATION (Part B)

Survey of Northeast Regional and Intercity Household Travel Attitudes and Behavior

INTRODUCTION

This is to request the Office of Management and Budget’s (OMB) one year approved clearance for the information collection entitled, “Survey of Northeast Regional and Intercity Household Travel Attitudes and Behavior”

Part B. Collections of Information Employing Statistical Methods

The proposed study will employ statistical methods to analyze the information collected from respondents and draw inferences from the sample to the target population. It will use an overlapping dual frame design with respondents interviewed on landline phones and on cell phones. The following sections describe the procedures for respondent sampling and data tabulation.

1. Describe the potential respondent universe and any sampling or other respondent selection method to be used.

1. Respondent universe

The respondent universe consists of all persons aged 18 to 74 residing in households with a working telephone located within the Northeast Corridor of the United States. The Northeast Corridor is defined graphically in Figure B.1. Also, since interviews will be conducted in only English, any person who does not speak English will be excluded from this study. Based on the prevalence of the use of English in the United States and especially among target respondents, who are long distance travelers, excluding respondents who speak a language other than English will have minimal impact on the study.

FIGURE B.1 NORTHEAST CORRIDOR GEOGRAPHY

The current population of each of the areas shown in Figure B.1, based on 2010 US Census, is summarized Table B.1.

TABLE B.1 NORTHEAST CORRIDOR POPULATION BY AREA

It is not anticipated that the use of a telephone household sample will cause coverage bias. Research has shown that telephone service for many households is intermittent, and that gains or losses in service are associated with changes in their financial situation, (Keeter, 1995¹). By implication, telephone coverage is not a static phenomenon, but rather, the telephone population at any given point includes households that were recently part of the non-telephone population. Households with intermittent telephone service are similar on a variety of important demographic characteristics to households who usually lack telephones (Keeter, 1995; Frankel et al, 2003²). Thus, weighting using information on interruptions in telephone service can ensure the correct coverage in the final sample of persons including those in non-telephone households.

2. Statistical sampling methods

A sample of 22,500 people will be selected into the study during Phase 1 in order to yield 15,000 completed follow-up surveys in Phase 2. The Phase 1 sample design will employ a partially overlapping dual frame design with probability samples drawn from independent sampling frames for landline phones and for cell phones.

The landline sample will be drawn from telephone banks randomly selected from an enumeration of the Working Residential Hundred Blocks within the active telephone exchanges. The Working Hundreds Blocks are defined as each block of 100 potential telephone numbers within an exchange that includes one or more residential listings (i.e., this will be a list-assisted sample). A two-digit number will then be randomly generated for each selected Working Residential Hundred Block to complete the phone number to be called.0 By randomly generating these numbers, a process known as random digit dialing (RDD), every number in the sampling frame of Hundreds Blocks has an equal probability of selection regardless of whether it is listed or unlisted. The RDD sample of telephone numbers is dialed to determine which are currently working residential household telephone numbers. The systematic dialing of those numbers to obtain a residential contact should yield a probability sample of landline telephone numbers.

The next step is selecting the household member to interview. Once a household is contacted, the interviewer will introduce him/herself and the survey, and ask how many persons residing in the household are 18 to 74 years old. If there is only one person in that age range residing in the household, then the interviewer will seek to interview that person. If there are multiple people in that age range residing in the household, the interviewer will randomly select one of those household members by asking to speak with the age-eligible household member who had the most recent birthday. Only that randomly selected person will be eligible to participate in the survey; no substitution by other household members will occur. It was decided to obtain data from only one person in each household though the intracluster (household) correlation is likely to be low. Collecting data for more than one person imposes an increased response burden. Also, the average number of persons over 18 years in a household was less than 2.59 in 2010 according the U.S. Bureau of Census. Therefore, it is unlikely that the variance of the estimates would increase substantially because of sampling one person within a household. If the selected respondent is not available, then the interviewer will arrange a callback. In cases where no one residing within the household is in the eligible age range, the interviewer will thank the individual who answered the phone, terminate the call, and the number will be removed from the sample.

Although list-assisted landline RDD sampling provides only a small coverage error for landline telephone households within landline banks, the restriction of the sampling frame to only landline banks would introduce a much more serious coverage error in general population surveys. The increasing percentage of households that have abandoned their landline telephones for cell phones has significantly eroded the population coverage provided by landline-based surveys. The key group that is missing from landline RDD samples is the cell phone-only group. But there is also potential bias in landline samples from under-coverage of young people who tend to rely on their cell phones more than their landline phones. The cell phone sample for this survey will therefore be composed of both cell phone only and cell phone users with a landline in their household.

Due to the higher cost of cell phone interviews compared to landline interviews, dual frame surveys are not usually designed with proportional allocation of sample between the landline and cell phone strata. The most recent data published from the National Health Interview Survey³ shows 23.1% of adults residing in cell phone only households in the Northeast region of the United States during the first half of 2012 The sample allocation for the Phase 1 sample will consist of a two-stratum design (landline and cell phone) with 22% of the total Phase 1 sample (or 4,950 interviews) being allocated to the cell phone sample.

The cell phone sample will be randomly selected from 1,000 banks used exclusively for cell phones, using RDD. We are not stratifying by area code since area code is not a great predictor of geography for cell phone sample. Procedures for sample selection will be similar to those used in selecting the landline sample, except that the sample will not be list-assisted as it is in the landline sample. In addition, the cell phone will be treated as a single user device. This means the cell phone sample will not require the procedures used with the landline sample to select from multiple eligible household members. Upon contact and introduction, the interviewer will immediately ask questions to determine that the person on the phone is not in a situation that could pose a safety risk to that individual (e.g., driving at the time of the call). If the contacted individual is found to be in a situation that could pose a risk, the interviewer will terminate the call and call back another time. If it is safe for the contacted individual to proceed with the call, then the interviewer will move on to asking screening questions regarding age eligibility.

3. Precision of sample estimates

The objective of the sampling procedures described above is to produce a random sample of the target population. This means that with a randomly drawn sample, one can make inferences about population characteristics within certain specified limits of certainty and sampling variability.

The margin of error, d, of the sample estimate of a population proportion, P, equals:

Where t_α equals 1.96 for 1-α = 0.95, and the standard error of P equals:

Where:

deff = design effect arising from the combined impact of the random selection of one eligible individual from a sample household, and unequal weights from other aspects of the sample design and weighting methodology, and

n = the size of the sample (i.e., number of interviews)

Using these formulas, the margin of error for a sample size of 15,000 interviews is d = 0.0098 to 0.0113, using an average deff of 1.5 to 2.0. and setting P equal to 0.50. The need for this sample size is driven by plans to examine key geographic and trip purpose subgroups, which is discussed later in this document. The Pew Research Center, the Kaiser Family Foundation and AP/GfK report average design effects for their overlapping dual frame sample designs of 1.39, 1.33 and 1.68, respectively (Lambert et al. 2010⁴), making it reasonable to assume that the average design effect will be around 1.5 to 2.0.

4. Sample weighting – Landline sample

Starting with the calculation of the weights for the landline RDD sample, the base

sampling weight will be equal to the reciprocal of the probability of selecting the household’s telephone number in its geographic sampling stratum. Thus, the base sampling weight will be equal to the ratio of the number of telephone numbers in the 1+ working banks for the number of telephone numbers drawn from those banks and actually released for use. Next, the reciprocal of the number of voice-use landline telephone numbers in the household (up to a maximum of three such numbers) will be calculated. It will compensate for a household’s higher probability of selection when it has multiple voice-use residential telephone lines. As noted earlier we will compensate for the exclusion of nontelephone households by using an interruption in landline telephone service adjustment. A within-household weighting factor will then be created to account for the random selection of one respondent from the household. The design weight then equals the product of each of the weight components described above.

Adjustment for unit non-response will be based on at least two cells. The non-response adjustment cells will be defined by whether the respondent (or person who was selected for an interview but did not complete it) resides in a directory-listed number household. Other telephone exchange variables may be considered in the development of the unit non-response adjustments. The non-response adjusted weight equals the design weight times the non-response adjustment factor.

5. Sample weighting – Cell phone sample

For the cell phone sample the first step is to assign a base sampling weight that equals the reciprocal of the probability of selection of the cellular telephone number. The cellular phone will be treated as a personal device so there will be no random selection of a respondent. The design weights will then be adjusted for unit non-response separately for cell phone-only respondents and for cell mostly respondents. Note that the control totals for the post-stratification adjustment will come from the NHIS. At this point we have non-response adjusted weights for the cell phone sample.

6. Sample weighting – Combined sample

The first step involves compositing the dual user respondents from the landline sample with the dual user respondents from the cell phone sample. The compositing factor, λ, will be calculated to minimize the variance of the estimate of the population total. The details are shown under the section on variance estimation.

In the final file used for weighting, all contacted households from the landline cross-section sample and the cell phone sample will be included. It is important to include all contacted households, rather than only completed interviews since the contacted households provide useful information for weighting purposes.

The population control totals for the raking of the combined sample will be based primarily on the latest Census Bureau population estimates (age by gender by race/ethnicity) and the American Community Survey (ACS) to develop some of the population control totals for socio-demographic variables such as education. Battaglia et al. (2008)⁵ have shown the potential for strong bias reduction in RDD samples from the inclusion of socio-demographic variables in the raking. We will also use the results of the nonresponse bias analysis to inform which variables should be included in the raking.

One might rake the combined sample to the socio-demographic control totals discussed above, but it is important to also have raking control totals related to telephone usage: number of persons age 18 to 74 years old that only have landline telephone service, number of persons age 18 to 74 years old that only have cellular telephone service, and number of persons age 18 to 74 years old that have dual (landline and cellular) telephone service. Such estimates can be obtained for the U.S. from the National Health Interview Survey (NHIS).

To develop the final weights, the non-response adjusted weights will be raked using gender by age, race/ethnicity, education, and telephone usage group as margins... The latest version of the Izrael, Battaglia, Frankel (IBF) SAS raking macro (Izrael et al. 2004)⁶ will be used to implement the raking. The raking macro will allow both raking and trimming of large weights. Median weight plus six times the interquartile range will be used as the limit for identifying large weights. The possibility of using additional margins will be examined.

7. Variance estimation

Variance estimation will need to rely on statistical software such as SUDAAN, however, existing software will not directly calculate standard errors for overlapping dual frame designs. Rather, it will be necessary to calculate the variance for each of the two samples and use the results to calculate the variance for the combined sample.

The dual frame estimator commonly used for overlapping dual frame telephone samples is:

LL designates the landline sample and CELL designates the cellular sample. LL-O, CELL-O, and DUAL identify the landline-only, cell-only, and dual service domains, respectively. λ is the compositing factor for the overlap component.

The composting factor that minimizes the variance of is given by

.

The estimates are based on a random sample of households in the landline frame and persons in the cellular fame. Therefore, thee variances are functions of effective sample sizes. The effective sample sizes are computed by taking the sample sizes (respondents) and dividing by the design effects of the estimates. The design effect is based on the coefficient of variation of the sampling weights in each sample. Let be the effective sample size of cell phone users from the landline fame and let be the sample size of persons having landline from the cell phone frame. Then, the value of is approximated by

.

If we treat λ as fixed, then the variance of can be written as:

_{The first component will be estimated for the landline sample using SUDAAN by taking the complex nature of the sample design into account. The second component will be estimated for the cellular sample also using SUDAAN by taking the complex nature of the sample design into account. SUDAAN will take into account both the sampling weights and the design used for the selection of the sample while computing variance estimates.}

2. Describe the procedures for the collection of information.

1. Call/Contact strategy

The NEC Future recruitment survey in Phase 1 will be administered via CATI to a randomly selected sample of 22,500 respondents age 18 to 74 years old residing in the Northeast Corridor who have made longer distance trips in the Northeast Corridor within the past 12 months. Approximately 17,550 interviews will be conducted with a landline sample, and 4,950 interviews conducted with a cell phone sample.

Interviewing will be conducted according to a schedule designed to facilitate successful contact with sampled households and complete interviews with the designated respondent within those households. Initial telephone contact will be attempted during the hours of the day and days of the week that have the greatest probability of respondent contact based on the call history of previous surveys conducted at Abt SRBI. This means that much of the interviewing will be conducted between 5:30 p.m. and 9:30 p.m. on weekdays; between 9:00 a.m. and 9:30 p.m. on Saturdays; and between 12:00 noon and 9:30 p.m. on Sundays.

The NEC Future recruitment survey will include daytime calling within its calling algorithm. Research has demonstrated that a variation in calling times can reach subsamples disproportionately. For the general population, evenings have the highest contact rate as opposed to weekdays and weekends. But for low income groups and minorities, daytime calls increase the probability of contacting the respondent. Lower income respondents are more likely to have irregular work hours (e.g. shift work, etc.) and sometimes have lower labor market participation, which places them at home during the day.

The NEC Future recruitment survey will employ a 5 call strategy for landline and cell phone numbers where up to 5 call attempts will be made to ringing but unanswered numbers before the number is classified as a permanent no answer. Callbacks to unanswered numbers will be made on different days over a number of weeks according to a standard callback strategy. If contact is made but the interview cannot be conducted at that time, the interviewer will reschedule the interview at a time convenient to the respondent. If someone picks up the phone, but terminates the call before in-house selection of a designated respondent can be made, then the Contractor will apply a protocol to maximize participation while minimizing respondent annoyance.

For Phase 2, the follow-up survey will be administered by web for those with Internet access. Individuals who choose to do the follow-up via web will be sent an email with a unique link to their individualized follow-up web survey. Those without Internet access will be mailed a packet of customized SP choice exercises. Respondents will be able to view the travel scenarios when they are called for the phone version of the follow-up survey. Respondents will be contacted for the follow-up phone survey in about 10 days; this length of time should be sufficient to customize the travel scenarios for each respondent and for respondents to receive the mailing.

2. Protocol for answering machines and voice mail

When contact is made with an answering machine or voice mail, a message will be left according to a set protocol. For landline numbers, a message will be left on the 3^rd attempt to contact a household member. The message will explain that the household had been selected as part of a national USDOT study, ask that the Contractor’s toll-free number be called to schedule an interview, and include reference to the FRA web site which will include information about the survey so that prospective respondents can verify the survey’s legitimacy. For the follow-up survey phone contacts that reach an answering machine or voice mail, a message will be left that we tried to reach them and that will offer the option of having them call our toll-free number to schedule an appointment.

3. Initiating the interview

When a household is reached in the landline sample on the recruit survey, the interviewer will screen for age eligibility. If only one household member is age eligible, then the interviewer will seek to interview that individual. If there is more than one eligible household member, then the interviewer will randomly select one respondent from among them and seek to interview that person. Appointments will be set up with respondents if it is inconvenient for them to be interviewed at the time of contact. If the randomly selected respondent is not available at the time of contact, then the interviewer will ask what would be a good time to call back to reach that person.

For the cell phone sample on the recruit survey, respondents will first be asked if they are in a situation where it would be unsafe to speak with the interviewer. If the respondent says “Yes,” then the interviewer will say that he or she will call back at another time, and immediately terminate the call. Once a cell phone user is reached at a safe time, the interviewer will first screen for age eligibility. If the cell phone user is eligible to participate, then the interviewer will seek to proceed with the interviewer. If it is an inconvenient time for the respondent, then the interviewer will try to set up an appointment.

The follow-up phone survey will first remind respondents that they had completed the recruit survey and agreed to do this follow-up survey. The interviewer will also ask respondents to retrieve the packet of travel scenarios for reference. If they did not receive it or have lost it, we will confirm the mailing address on file and resend as necessary.

4. Interviewer monitoring

Each interviewer will be monitored throughout the course of the project. The Monitor will evaluate the interviewer on his or her performance and discuss any problems that an interviewer is having with the Shift Supervisor. Before the end of the interview shift, the Monitor and/or Shift Supervisor will discuss the evaluation with the interviewer. If the interviewer cannot meet the Contractor’s standards, he or she will be removed from the project.

All interviewers on the project will undergo two types of monitoring. The Study Monitor will sit at a computer where s/he can see what the interviewer has recorded, while also audio-monitoring the interview. The audio-monitoring allows the Supervisor to determine the quality of the interviewer's performance in terms of:

1. Initial contact and recruitment procedures;

2) Reading the questions, fully and completely, as written;

3) Reading response categories, fully and completely, (or not reading them) according to study specifications;

4) Whether or not open-ended questions are properly probed;

5) Whether or not ambiguous or confused responses are clarified;

6) How well questions from the respondent are handled without alienating the respondent;

7) Avoiding bias by either comments or vocal inflection;

8) Ability to persuade wavering, disinterested or hostile respondents to continue the interview; and,

9) General professional conduct throughout the interview.

The Supervisor will also monitor the interviewer's recording of survey responses as the Supervisor's screen emulates the interviewer's screen. Consequently, the Supervisor can see whether the interviewer enters the correct code, number or verbatim response.

5. Survey content and structure

As described in detail in Supporting Statement A, Question #2, the survey includes questions about current longer distance travel and trips as well as Stated Preference (SP) questions about alternative choices in response to different mode availability and modal service characteristics. Respondents should be able to recall the details of these less frequent and likely more memorable long distance trips.

The specific SP trade-off questions reflect an experimental design that addresses a cross section of all of the potential mode availability and service characteristic combinations so that each respondent is not asked to address too complex a choice task or is unnecessarily burdened by a longer interview. Specifically, each respondent is presented with choices questions addressing three of the following six (6) modes of travel within the NEC:

• High Speed Train

• Regional Train

• Commuter Train

• Commuter Train connecting with a Regional Train

• Passenger Car/Truck/Van

• Plane

• Bus

• For a specific respondent, the selected modes used in the stated preference questions will include the respondent’s first choice mode, (i.e., the mode they currently use for travel in the NEC), the respondent’s second choice mode (if their first chosen mode was not available), and a randomly selected third choice mode. Thus, each respondent will be asked to choose from among three modes of travel. Based on the prior the 2006 Amtrak NEC Traveler Surveys, which also included questions about second choice modes, and the ACS CTPP Journey to Work (JTW) flow data, which provide a basis for estimating the incidence of longer commuter trips, we have estimated the expected number of respondents that will be exposed to each of the NEC modes is as follows: High Speed Train: 6,600 respondents

• Regional Train: 9,400 respondents

• Commuter Train: 3,200 respondents

• Commuter Train connecting with a Regional Train: 3,200 respondents

• Passenger Car/Truck/Van: 13,300 respondents

• Plane: 4,900 respondents

• Bus: 4,900 respondents

Note that the above total to 45,000, or three times the sample size, because each respondent will be exposed to three modes.

The detailed trip information obtained before the stated preference trade-off questions provide the context for the respondent’s travel choices and a basis for defining trip-relevant service characteristics in the trade-off questions. The stated preference questions vary the values of a randomly selected subset of service characteristics for each mode using an experimental design that minimizes the correlation among independent variables. Respondents will be randomly assigned to one of five (5) subgroups who will see changes in these variables:

• Group 1: travel time and cost

• Group 2: travel time and schedule interval (use travel time and cost for auto)

• Group 3: cost and schedule interval (use travel time and cost for auto)

• Group 4: travel time and reliability (likelihood of delay)

• Group 5: cost and reliability (likelihood of delay)

This survey design limits the number of changing variables that any one respondent will need to react to and thus makes the task more manageable. Across the entire sample, however, all of the above combinations will be addressed. Taken in combination with the expected respondent samples by mode, the smallest cells would contain 600 respondents in each of the above groups exposed to a Commuter Train connecting with a Regional Train mode choice.

Respondents will be presented with a total of twelve (12) SP questions that addresses three choices of mode of travel with varying characteristics within one of the five pairs of variables listed above. Thus, there will be a total of 6 variables, 2 for each of the three modes, and the experimental design will change the variables as follows across these 12 questions:

SP	Mode 1		Mode 2		Mode 3
Question	Variable 1	Variable 2	Variable 3	Variable 4	Variable 5	Variable 6
1	[V1-Base]	[V2-Base]	[V3-Base]	[V4-Base]	[V5-Base]	[V6-Base]
2	[V1-High]	[V2-High]	[V3-Base]	[V4-Base]	[V5-Base]	[V6-Base]
3	[V1-Low]	[V2-Base]	[V3-Low]	[V4-Low]	[V5-Base]	[V6-Base]
4	[V1-Base]	[V2-High]	[V3-Low]	[V4-Base]	[V5-Base]	[V6-Base]
5	[V1-High]	[V2-Base]	[V3-Base]	[V4-Low]	[V5-Base]	[V6-Base]
6	[V1-Low]	[V2-High]	[V3-Base]	[V4-Low]	[V5-Base]	[V6-Base]
7	[V1-Base]	[V2-Base]	[V3-Base]	[V4-Base]	[V5-Low]	[V6-Low]
8	[V1-High]	[V2-High]	[V3-Base]	[V4-Base]	[V5-Low]	[V6-Low]
9	[V1-Low]	[V2-High]	[V3-Base]	[V4-Base]	[V5-Base]	[V6-Base]
10	[V1-Base]	[V2-High]	[V3-Base]	[V4-Base]	[V5-Base]	[V6-Low]
11	[V1-High]	[V2-Base]	[V3-Base]	[V4-Base]	[V5-Base]	[V6-Base]
12	[V1-Low]	[V2-High]	[V3-Base]	[V4-Base]	[V5-Low]	[V6-Base]

The above experimental design offers three levels of the first variable and two levels of the remaining variables (a 3x2x2x2x2x2 design). It represents a compromise between the desire for a lot of variation and the need to keep to a reasonable number of questions and minimize the correlation among the variables. The ordering and assignment of modes and variables to this experimental design will, however, be randomized so each mode and variable combination will have the opportunity to appear in the first column and receive three levels of variation.

Base values of all variables are pre-determined for each mode for each possible origin and destination in the market area. The values for a particular respondent are determined by the respondent’s self-reported origin and destination from the randomly selected “reference trip” (see Questions 2 & 3 in Screener Survey for phone respondents and Questions 2 & 7 in the Follow-up Survey for internet respondents) as well as respondent-provided fare information (see Questions 5A-5C in Screener Survey for phone respondents and Questions 12A-C & 13A-B in the Follow-up Survey for internet respondents; If respondent does not remember the fare paid, default values will be used based on published fares for travel by the chosen mode between the origin and destination). The project team has developed transportation network models of all modes which produce the relevant base values for travel times and costs, which scale with trip length and geography. High and low values will be computed as follows:

• Total Travel Time High: randomize among +15%, +30% over base values

• Total Travel Time Low: randomize among -15%, -30% under base values

• Total Cost High: randomize among +15%, +30% over base values

• Total Cost Low: randomize among -15%, -30% under base values

• Delay High: randomize among +25%, +50% over base values

• Delay Low: randomize among -25%, -50% under base values

• Schedule High: randomize among +15 minutes, +30 minutes, +45 minutes over base values

• Schedule Low: randomize among -15 minutes, -30 minutes (only if base > 30 minutes), 45 minutes (only if base > 45 minutes)

The responses to these above SP survey questions ultimately provide the basis for estimating key sensitivities to changes in the service characteristics in the new model. This is described in further detail Supporting Statement A.

Considerable care has been taken the design of the NEC FUTURE survey to avoid presenting or showing any bias for/against any specific mode to respondents. First and foremost, the survey is household-based and mode neutral in recruiting respondents and identifying specific trips they have taken. Key elements of this include:

• Identification of the survey as USDOT-sponsored, not FRA, when speaking with respondents

• Randomized order of presenting candidate markets to select a specific market where the respondent made a trip

• Random selection of specific mode and trip purpose trip taken in this market

• Use of this randomly selected mode as one of the available modes shown in the stated preference questions

• Using respondent’s reported alternative mode (second choice) as one of the available modes shown in the stated preference questions

• Randomly selecting from remaining available modes to complete the list of modes shown to each respondent in the stated preference questions

• Randomizing the order that these modes are shown to respondents

• Using similar language throughout the survey where mode-specific information is presented to respondents

In addition, the FRA has provided briefings on the survey approach during expert panel meetings of an OST-sponsored, FHWA-managed long distance travel demand modeling project. FHWA, FAA, FTA and BTS are all participants in the project, which FRA has been involved in since 2011. FHWA has specifically requested and reviewed the draft survey instrument, and has pursued active discussions with FRA on how to gain the most DOT-wide value from FRA’s current efforts. We fully anticipate sharing data and results from this travel survey with other modes, both for Northeast Corridor specific understanding, and also to inform national travel demand models

NEC FUTURE study includes additional outreach to constituencies representing both rail and non-rail modes of travel within the NEC. Non-rail agencies on the stakeholder correspondence list include:

• Federal Highway Administration (headquarters and regions within the NEC)

• Federal Aviation Administration (headquarters and offices within the NEC)

• Federal Transit Administration (headquarters and region offices within the NEC)

• Bradley International Airport

• Islip MacArthur Airport

• Newcastle County Airport

• Rhode Island Airport Corporation

• T F Green Airport

• Tweed New Haven Regional Airport Maritime Administration

• Delaware River & Bay Authority

• Delaware River Joint Toll Bridge Commission

• Maryland Transportation Authority

• Massachusetts Department of Transportation

• Metropolitan Transportation Authority - Bridges and Tunnels

• New Jersey Turnpike Authority

• New York State Bridge Authority

• New York State Thruway Authority

• Pennsylvania Turnpike Commission

• Rhode Island Turnpike and Bridge Authority

• Port Authority of New York & New Jersey

• New Haven Parking Authority

• Delaware River Port Authority

• Diamond State Port Corporation (Port of Wilmington)

• MassPort (Port of Boston)

• New Haven Port Authority

• Philadelphia Regional Port Authority

6. Survey sample requirements

The key dimension driving the survey sample size requirements is trip purpose. Prior survey research and model estimation analysis, including the previously referenced Amtrak survey and model as well as most other intercity and regional surveys/models nationwide, have consistently shown trip purpose to be a significant determinant of travel behavior with respect to key sensitivities to service characteristics. For example, travelers on business trips typically show a higher value of time than travelers on non-business trips. As such, the new model will be stratified into the following three trip purposes:

• Commute Trips, including only the daily commute to or from the usual place of work

• Business Trips, which include all non-commute trips associated with a business purpose such as company meetings, sales trips, etc.

• Non-Business Trips, which includes all other non-commute and non-business trips

Another important dimension is geography. The survey sample needs to address a cross-section of trips in different markets that is representative of the NEC. In the NEC Future study, initial review and analysis of available market data and conceptual future rail alternatives has identified and confirmed the importance of the following key geographic stratification of the NEC:

• Travel North of New York

• Travel South of New York

• Travel through the New York Area (between points north of and south of New York)

The above stratification is particularly important with respect to business and non-business “intercity” trips, where there are important differences in the characteristics and availability of different modes of travel. The specific type of longer commute trip between regions addressed by the new model is in itself a unique market. However, it a much smaller market that does not lend itself to similar geographic stratification, particularly with respect to the “Through NY” segment, which is a very small market.

Since the new model will be stratified into business, non-business and commute trips, it is important to ensure the overall survey sample provides sufficient numbers of completed surveys within each of these segments. It is a given that larger samples will provide more precision, but there is a diminishing return on this relationship and requirements must be properly balanced with the need to efficiently and effectively use available resources.

As with any sampling plan, there is always some uncertainty as to whether the data will actually reflect the universe of travelers until the survey itself is completed. However, there are available sources of data to estimate what could be reasonably expected from an NEC sample. Two key sources were examined to assess the expected distribution of the NEC sample – (1) the 2006 Amtrak NEC Traveler Surveys, which used a similar survey approach, and (2) the 2006 to 2008 3-year ACS CTPP Journey to Work (JTW) flow data, which provide a basis for estimating the incidence of longer commuter trips between NEC regions. Table B.2 below provides the estimated size of the key trip purpose and geography subsamples for within a total sample of 15,000 completed follow-up surveys (expected from 22,500 recruits).

TABLE B.2 FOLLOW-UP SURVEY SAMPLE BY TRIP PURPOSE & GEOGRAPHY

	Business	Non-Business	Commute	Total
North of NY	620	2,955	435	15,000
South of NY	1,490	6,980	390
Through NY	445	1,640	45*
Total	2,555	11,575	870

* this subgroup is not a significant contributor to travel in the NEC and it will not be examined independently

The margin of error at 95% confidence level for an estimated population percentage of 50% based on the sample sizes given under Business in Table B.2 will be between 2.4 percentage points and 5.7 percentage points under the assumption that the design effect is 1.5. The margin of error at 95% confidence level for non-business will be between 1.1 and 3 percentage points.

Another important consideration is the intersection of the survey experimental design with trip purpose. Table B.3 below shows the expected number of respondents that will be exposed to each of the NEC modes by trip purpose, using the same Amtrak survey data sources. Again, note that Table B.3 totals to 45,000, or three times the sample size, because each respondent will be exposed to three modes.

TABLE B.3 FOLLOW-UP SURVEY RESPINDENT EXPOSURE TO MODES BY TRIP PURPOSE

	Business	Non-Business	Commute	Total
High Speed Train	1,300	5,020	280*	6,600
Regional Train	1,705	7,350	345	9,400
Commuter Train	470	2,070	660	3,200
Commuter Train connecting with a Regional Train	535	2,445	220 **	3,200
Passenger Car/Truck/Van	1,890	10,560	850	13,300
Plane	1,060	3,840	***	4,900
Bus	705	3,440	255	4,400
Total	7,665	34,725	2,610	45,000

* Because of its limited-stop operation, high-speed train service (like Amtrak’s Acela) is not available in many commuter markets

** Mixed rail trips, which include a transfer, tend to serve longer trips beyond the commuter shed

*** Plane service does not exist in most commuter markets; it is not a relevant mode of travel for commuters because of price.

Based on the available data, the above subsamples can be achieved from a random sample of the NEC population without any oversampling. Both business and non-business trips can be easily segmented geographically, and have separate targets based on the NEC geography. With one exception, these subsamples would also be achieved with random selection of a specific respondent trip among those reported. The exception is for the commute purpose, where the very low incidence of this trip requires selecting that particular trip from among all respondents who report it.

7. Quality Assurance and Quality Control of Data

The JV and Abt SRBI, the JV’s market research contractor, will implement a comprehensive quality assurance and quality control system to ensure the delivery of a clean database with a maximum degree of consistency, completeness and accuracy.

The process begins with the software Abt SRBI uses for telephone and web based surveys. Our CATI and web software system program loops, rotations, randomization and extremely complex skip patterns. It also includes automatic range checks for data entry. It can be programmed to conduct complicated calculations. And it can also carry forward earlier responses, which can be integrated into later questions. All these functions ensure the quality of the data collected.

In addition to programming, ad-hoc manual inspections of collected data will be conducted as further checks. This primarily includes doing sense checks on the frequencies of questions, and examining the distribution of key survey variables (e.g., chosen mode).

Once initial data collection is underway and 300 responses are obtained, they will be preliminarily tested for general operational and content issues with the survey as well as respondent fatigue in the stated preference questions. These initial 300 responses constitute the “pilot phase” of the survey, as discussed later in this document.

Specific testing of the initial responses, primarily the stated preference questions, is important to undertake to ensure the responses will provide a solid basis for model estimation. Items to be tested include:

• Do respondents demonstrate enough willingness to switch modes in the SP questions to support model estimation. If not, we may need to increase the level of variation in the level of service variables.

• Prevalence of illogical responses, such as respondents switching modes under less desirable circumstances. If this is prevalent, we may need to simplify the SP exercises.

• Once initial checks on respondent mode switching behavior are satisfied, test model estimation will be performed as a more extensive check on whether the SP responses will support model estimation. While the test models will not be expected to be statistically significant due to the low number of records, they will indicate whether key variables such as time and cost are providing meaningful coefficients. If not, the level of service variables presented in the survey may need to be adjusted.

The respondent fatigue may increase with the number of choice experiments presented to each respondent and the number of alternatives available and the number of attributes presented for each alternative in each experiment. The level of complexity the SP questions and the number of SP questions ask of an individual respondent is a matter of judgment. The decision is a tradeoff between the additional effort required on the part of the respondent with each additional experiment and the value of the additional information obtained. In this study, a decision has been made to include 12 experiments with three alternatives, each with two attributes that vary and two that are fixed. The decision to adopt three alternatives each with two attributes that vary is believed to be a reasonable and desirable design.

As a preemptive measure to mitigate the possible impacts from respondent fatigue, the stated preference questions will be structured so that assignment of modes and variables is random. For example, the first eight questions vary characteristics of two modes, holding the third constant, and the last four questions vary the characteristics of the first and third modes, holding the second constant. The assignment of the respondent’s second choice mode and a randomly chosen mode to the second and third positions will be randomized, so they will have equal chance of being the focus of the early or later stated preference questions. Therefore, if we determine that some of the later responses should not be used in model estimation due to respondent fatigue affecting quality of responses, the integrity of the experimental design will not be comprised. That is, we will have adequate observations where there is variation for both the respondent’s stated “second choice” mode and the randomly assigned mode.

Furthermore, after we have begun to conduct the survey and have data from 300 respondents, we will count of the number of experiments declined (no response provided) for experiments 1 through 12. We will also measure the time spent responding to each experiment 1 through 12. Both of these checks will provide an indication if respondents are finding it increasingly difficult to complete the survey.

If any issues are identified with either the survey content and operation or respondent fatigue, the survey instrument will be revised as needed. The primary areas which could require revision would be the specific variable values in the stated preference questions as well as the ordering of the stated preference questions. These conditions would primarily impact the experimental design for the stated preference questions.

3. Describe methods to maximize response rates and to deal with issues of non-response.

1. Initial contact

The initial contact with the designated respondent is crucial to the success of the project. Most refusals take place before the interviewer has even completed the survey introduction (usually within the first 30 seconds of the call). Numerous studies have shown that an interviewer's manner of approach at the time of the first contact is the single most important factor in convincing a respondent to participate in a survey. Many respondents react more to the interviewer and the rapport that is established between them than to the subject of the interview or the questions asked. This positive first impression of the interviewer is key to securing the interview.

While the brief introduction to the study concerning its sponsorship, purpose and conditions are sufficient for many respondents, others will have questions and concerns that the interviewer must address. A respondent's questions should be answered clearly and simply. The interviewers for the NEC Future Survey will be trained on how to answer the most likely questions for this survey. A hard copy of Frequently Asked Questions will be provided to each interviewer at their station (see attached file NEC FUTURE FAQ’s). The interviewers also will be trained to answer all questions in an open, positive and confident manner, so that respondents are convinced of the value and legitimacy of the study. If respondents appear reluctant or uncertain, the interviewer will provide them with a toll-free number to call to verify the authenticity of the survey. Above all, the interviewer will attempt to create a rapport with the study subject and anyone in the household contacted in the process of securing the interview.

2. Refusal documentation and tracking

Higher response rates can be achieved through procedures built on careful documentation of refusal cases. Whether the initial refusal is made by the selected respondent, a third party or an unidentified voice at the end of the phone, the CATI system used for this survey will be programmed to move the interviewer to a non-interview report section. The interviewer will ask the person refusing to tell him/her the main reason that he or she doesn’t want to do the interview. The interviewer will enter the answer, if any, and any details related to the refusal into the program to assist in understanding the strength and the reason for the refusal. This information, which will be entered verbatim in a specific field in the CATI program, will be part of the formal survey reporting record, and will be reviewed on a daily basis by the Field Manager and, on a weekly basis, by the Project Director.

The Survey Manager and the Project Director will review the information about refusals and terminations on the CATI system on an ongoing basis to identify any problems with the contact script, questionnaire or interviewing procedures that might contribute to non-participation. For example, they will scrutinize the distribution of refusals by time of contact, as well as any comments made by the respondent, in order to determine whether calling too early or too late in the day is contributing to non-participation. Also, the refusal rate by interviewer will be closely monitored.

In addition to relying on the CATI data records, the Project Director and Survey Manager will also consult with the interviewing Shift Supervisor, who has monitored the interviewing and debriefed the interviewers. The information from these multiple sources provides solid documentation of the nature and sources of non-response.

3. FRA web site information

It is not uncommon for surveys such as these to generate incoming calls or emails to the U.S. Department of Transportation from people who have been contacted by the interviewer to conduct the survey. In a large percentage of the cases, it’s an inquiry to check its legitimacy. People are wary about attempts to extract information from them, wondering if the caller is truthful in identifying the source of the survey and the use of their information.

To help allay these fears, FRA will place on its web site information that prospective respondents can access to verify the survey’s legitimacy. The interviewers will provide concerned respondents with the web address for FRA’s home page. There will be a link on the home page that will direct the respondents to information on the source of the survey and why it is important for them to participate. An 800 number to reach the survey Contractor will also be provided so they can schedule an interview. This procedure was used with the 2007 Motor Vehicle Occupant Safety Survey. More than 700 people who participated in that survey indicated in an ending question that they had gone to the NHTSA web site to get more information about the survey, suggesting that this was an effective approach to raising the response.

4. Non-response bias analysis

A non-response bias study will be conducted for the NEC Future Survey. Non-response is the failure to obtain survey measures on all sampled individuals. There are two main potential consequences from non-response: (1) non-response bias and (2) underestimation of the standard errors. The main challenge posed by non-response is that without a separate study (i.e., a non-response bias analysis), one can never be sure of whether the non-response has introduced bias or affected the estimation of the standard errors. The mere presence of non-response, even high non-response, does not necessarily imply bias or non-response error.

The demographic composition of the NEC Future Survey landline sample will be compared to the demographics of non-respondents as determined by auxiliary data providing demographic information for a proportion of sampled landline households. The auxiliary data will be purchased from an external vendor like Marketing Systems Group (MSG) or Survey Sampling International (SSI) and will represent the latest and most recent information on demographic characteristics such as household income, presence of children in the household, number of people residing in the household, race of those living in the household, and age and gender of household members. The quality of this information varies since most of this is gathered though credit bureaus and applications for services. The younger and more mobile population is not as likely to be represented in the auxiliary data. Neither are minorities or lower income folks. This data does, however, give us a more robust analysis of who completed and who did not at the case level, rather than at the aggregate level, which is undertaken when comparing Census data to the sample.

The auxiliary data will be used in two different ways. First, the auxiliary data will be compared to the demographic data obtained from the surveys for those respondents who completed the interview. The analysis will assess whether the landline non-respondents are demographically different from those who completed interview. This will be done for both Phase 1 and Phase 2. If some of these characteristics are correlated with the propensity to respond, there is a chance that there might be some bias in those estimates. Secondly, the auxiliary data for survey respondents will also be compared to the auxiliary data for non-respondents. Again, this comparison will be done for be done for both Phase 1 and Phase 2.

Travel characteristics collected in the recruit survey will also be used to analyze non-response. Specifically, this data will be used to compare all who completed the recruit survey with the subset of respondents who also completed the follow-up survey.

Additionally, there will be a non-response follow-up (NRFU) survey with a sub-sample (n=500) of households that responded during Phase 1, but failed to respond to the Phase 2 interview. The NRFU has two main objectives: 1) investigate the reasons for non-response, and 2) assess the risk of non-response error to model estimates. Because these are people who refused to participate in Phase 2 there are limitations on how many respondents one can expect to participate in the NRFU. However, the sample size of 500 households is large enough to estimate factors that contributed to non-response, assess the potential for non-response bias in key substantive variables, and determine key socio-demographic variables to include in the final post-stratification adjustments.

The NRFU features an abbreviated survey instrument in which the household is asked to provide a few basic socio-economic and demographic variables, reasons for non-participation, and responses to a few basic survey items. NRFU households will each be offered $20 for their participation, as this amount has been effective in past non-response follow-up studies conducted by the Contractor. The results of the non‐response follow‐up survey will provide valuable information for assessing the extent to which non-response bias affects the overall survey respondent sample and for developing weighting adjustments that minimize the impact of non-response bias on sample statistics and model parameters.

The analysis will include comparison of key variables from the NEC Future Survey respondents with the same variables for the NRFU respondents. This will provide insights about the direction and magnitude of possible non-response bias. The project team will investigate whether any differences remain after controlling for major weighting cells (e.g., within race and education groupings). If weighting variables eliminate any differences, this suggests that weighting adjustments will reduce non-response bias in the final survey estimates. If, however, the differences persist after controlling for weighting variables, then this would be evidence that the weighting may be less effective in reducing non-response bias.

Based on experience with similar prior surveys, this study is expected to have a 25% to 30% response rate at Phase I and a 67% response rate at Phase II. We expect the follow-up response rates to be similar for both the internet and mail/phone versions. As such, the cumulative response rate across both Phase I and Phase II is expected to be 17% to 20%. We expect a 67% response rate based upon the outcome of two comparable two-phase studies conducted for Amtrak where respondents were recruited by phone and asked to complete the follow-up by web (or be willing to complete the follow-up by phone when web access was not possible) for a token appreciation of $5. In 2011, the follow-up rate was 67% using a 9 minute follow-up survey instrument and up to 3 reminders for completion. In 2007, the follow-up rate was 60% using a 10 minute follow-up survey instrument and up to 3 reminders for completion. In this study, the follow-up survey is 15 minutes with more reminders – up to 5 reminders for completion. Finally, please note that Web was chosen as the preferred follow-up mode as it is an efficient way for visual examination of the travel scenarios.

Table B.4 below summarizes the expected response rates for different sections of the survey. We will count a survey as a “complete” if the respondent provided a response to at least one stated preference question.

TABLE B.4 SURVEY QUESTION ITEM NON-RESPONSE ESTIMATES

4. Describe any tests of procedures or methods to be undertaken.

1. Testing the programming

The project team will test the CATI program thoroughly in test mode – running the interviewing program through multiple loops. The analytical staff will attempt to test all possible response categories for each question in order to identify embedded logic errors, as well as obvious skip problems. Several analysts may test the program simultaneously to identify problems quickly, and to double check the comprehensiveness of the testing.

2. Survey pretest

Pre-test interviews will be conducted with 9 respondents prior to the formal administration of the survey in order to test all of the survey systems, the survey instrument, and the CATI programming. The pretest will be conducted under conditions identical to the planned conditions for the final administration of the survey. It will be used to evaluate the wording of questions and response categories in terms of clarity and confusion for respondents; assess the flow of the interview, including question sequencing and skip patterns; check the time required to administer the questionnaire to see if it differs from the original estimate; assess item non-response to see if there are problems with particular questions; ensure that a smooth interview occurs with previous information guiding later questions; ensure random rotation of specified questions; check that invalid responses are not entered; and review and evaluate the full range of procedures.

Immediately following the interviews, the interviewers will participate in a debriefing session to report any problems or observations they have regarding the instrument to the field manager. The field manager will consolidate the observations and suggestions along with his own recommendations. These will be returned to the project director along with the hard copy of the pretest interviews. Interview timings will be provided as well. The completed questionnaires will be analyzed, along with the extensive comments on the content and administrative aspects of the instruments made by the interviewers, monitors and research staff observing the pretest. Revisions to procedures, CATI programming, and the survey instrument will be made as appropriate.

Since the JV and Abt SRBI, the JV’s market research contractor, have prior experience with similar travel survey designs that collect RP and SP data to support model estimation and development, we do not believe a formal independent pilot survey effort is necessary. We have already invested significant and we believe sufficient time and resources into the development of the survey, including a thorough review of the model and survey design with a Technical Working Group (TWG) representing FRA, Volpe, FTA, Amtrak, several commuter rail operators, consultants and academics with broad experience across different modes of transportation. Subsequently, the model and survey design were reviewed again in detail with BTS and OMB. Throughout these extensive reviews, the survey has undergone several rounds of refinement and we are confident that the final product will deliver the data needed to estimate a new model for NEC FUTURE. Furthermore, we are concerned that the additional time required to conduct a full independent pilot survey effort will delay the completion of the resulting travel demand model product such that it will not be available to fully serve the NEC FUTURE study to support the analysis and evaluation of future transportation alternatives within the schedule required by FRA.

However, as stated earlier, the survey effort will include an initial sample of 300 respondents constituting a “pilot phase”. After we have collected survey data from 300 respondents, we will suspend field work and conduct a thorough review and analysis of these completed data. The results of this “pilot phase” will be submitted to OMB for review. In submitting the results, we will provide information regarding: a summary assessment of how successful the pilot was and what changes are being proposed for the full-scale study, along with performance metrics from the pilot, such as: sample yield compared to expectations, screening and follow-up survey response rates, item response rates for key questions, consistency between key questions that would indicate that the respondents are providing usable responses, any observations that need to be discarded due to inconsistencies or any other indicators of quality of data.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design.

The following individuals have reviewed technical and statistical aspects of procedures that will be used to conduct the NEC Future Survey:

Bruce Williams

Senior Consulting Manager

AECOM Transportation

2101 Wilson Boulevard, 8th Floor

Arlington, VA 22201

(703) 340-3078

Mindy Rhindress, Ph.D.

Senior Vice President

Abt SRBI, Inc.

275 Seventh Ave, Ste 2700

New York, NY 10001

(212) 779-7700

Paul Schroeder

Vice President

Abt SRBI, Inc.

8403 Colesville Road, Suite 820

Silver Spring, MD 20910

(301) 608-3883

K.P. Srinath, Ph.D.

Senior Sampling Statistician

Abt SRBI, Inc.

(301) 628-5527

1 Keeter, S. Estimating non-coverage bias from a phone survey. Public Opinion Quarterly, 1995; 59: 196-217.

2 Frankel M, Srinath KP, Hoaglin DC, Battaglia MP, Smith PJ, Wright RA, Khare M. Adjustments for non-telephone bias in random-digit-dialing surveys.- Statistics in Medicine, 2003: 22; 1611-1626

3 Blumberg, Stephen J. and Julian V. Luke. Wireless Substitution: Early Release of Estimates From the National Health Interview Survey, July – December 2011. U.S. Department Of Health And Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics. Released 06/2012.

4 Lambert D, Langer G, McMenemy M. 2010. “CellPhone Sampling: An Alternative Approach”

Paper presented at 2010 AAPOR Conference, Chicago IL.

5 Battaglia, M.P., Frankel, M.R. and Link, M.W. 2008. Improving Standard Poststratification Techniques for Random-Digit-Dialing Telephone Surveys. Survey Research Methods. Vol 2, No 1.

6 Izrael, D., Hoaglin, D.C., and Battaglia, M.P.. 2004. “To Rake or Not To Rake Is Not the Question Anymore with the Enhanced Raking Macro “ Proceedings of SUGI 29 2004, Montreal, Canada

27