PRA-2126NEW.ACVDS.WC.SS.PartB.062410.Use[1]

Department of Transportation

Office of the Chief Information Officer

SUPPORTING STATEMENT

Annual Commercial Motor Vehicle Driver Survey:

Work and Compensation

This Supporting Statement is developed to request the Office of Management and Budget’s (OMB) review and approval of a new information collection (IC) entitled, “Annual Commercial Motor Vehicle Driver Survey: Work and Compensation.”

Part B. Collections of Information Employing Statistical Methods.

1. Describe potential respondent universe and any sampling selection method to be used.

The FMCSA is currently estimating that there are 7 million commercial motor vehicle drivers. There is no precise estimate regarding the number of drivers nor is there is a central nationwide source of information describing the population of drivers holding a commercial drivers license (CDL). Consequently, this information collection cannot claim to obtain input from a representative sample of drivers. Rather, the goal of this survey is to develop a methodology for obtaining information from CMV drivers and to develop a process for an annual survey of this population. The focus of this pilot study will be to test two different sampling methodologies (a mail survey and interviews) in two states (one in the east and one in the west). The current responses can be used to build a general understanding of this population, but FMCSA acknowledges that the data from this pilot study will be limited and therefore it does not intend to evaluate policy through rigorous statistical analyses of the survey results.

The two sampling methodologies have different trade-offs. Mail surveys offer the best potential for identifying a representative sample of CMV drivers(e.g., through a comprehensive, nationwide source of commercial motor vehicle driver addresses), so that drivers could be randomly selected for participation. Unfortunately, the FMCSA does not possess a database of driver addresses. This information is collected by organizations, such as the American Association of Motor Vehicle Administrators (AAMVA), and industry groups (e.g., Owner-Operator Independent Drivers Association (OOIDA), American Trucking Association (ATA), state chapter ATAs, Truckload Carrier Association (TCA)), but the information may not be readily shared to a third party due to privacy concerns. As a result, mail surveys are generally conducted in partnership and/or with assistance from carriers or industry groups. This method of sampling may introduce a bias in the selection process, however, and the results could reflect carrier-level differences. Consequently, the information collected may not be generalizable to the commercial motor vehicle driving population as a whole.

For the mail survey, two different sampling frames will be evaluated. In one, a sample of drivers will be identified by collecting driver addresses from the American Association of Motor Vehicle Administrators (AAMVA). AAMVA is a nonprofit organization that serves to develop programs to improve driver safety. AAMVA possesses a registration database from each State that includes drivers with current CDLs. Each State follows different reporting requirements for updating the information in their databases; in general, information from drivers is requested between 15 to 30 days of an address change. Some demographic information is also contained in the database, but this information varies from state to state.

The second mail-in methodology is to develop a partnership with a trucking organization and collaborate to conduct the survey. Specifically, the FMCSA will also partner with Owner-Operator Independent Driver’s Association (OOIDA) to identify another set of drivers for the mail survey. OOIDA’s membership database contains information for 157,000 owner-operators and 16,000 company drivers, and they would be willing to distribute the survey to a subset of their members.

The most common methodology for surveying CMV drivers is interviews¹. Drivers are generally interviewed at rest stops, truck stops, terminals, weigh stations, or inspection sights. However, one problem with intercept surveys is that the number of locations where drivers may be interviewed is generally limited (e.g., by resource or budget constraints). Consequently, selection of the intercept location introduces a bias into the survey design. It is important to understand how the composition of the population that frequent a particular location are similar to (or different from) the population that do not frequent that location, and how these similarities and differences will affect the interpretation and generalizability offered by the results.

For this study, three different sampling locations will be evaluated – Department of Motor Vehicles (DMV) offices, truck stops, and weigh stations. For each of these sites, the expected population and the potential coverage bias will be measured or estimated, as data allows. Interviews would be conducted on weekdays and on at least one weekend day. Interviewers would be researchers rather than enforcement staff. Researchers would coordinate with DMV offices, truck stops, and POE weigh stations to arrange the visit.

2. Describe procedures for collecting information, including statistical methodology for stratification and sample selection, estimation procedures, degree of accuracy needed, and less than annual periodic data cycles.

This pilot study will employ a 2 (survey administration: mail in, interviews) x 2 (incentive vs. no incentive) design. The purpose of this information collection is to understand the best methodology approach for reaching the CMV driver population as well as how to motivate them to respond. Because this information collection is intended to serve as a pilot study only, data will be collected in only two states. In each state, 500 CMV drivers will be sampled – 200 CMV drivers via a mail-in survey and 300 CMV drivers via interviews. To understand the use of incentives, half the participants will be offered a $10 incentive for participating, and the other half will not. Commercial vehicle drivers are typically paid by the mile and therefore any time spent not driving is money lost. Thus, it is of interest to examine whether the use of incentives will significantly increase drivers’ interest in, and participation in the survey.

The dependent variables will be the response rate, the quality of the responses, and the demographics of the respondents. The response rate is simply the ratio of the number of participants relative to the total number of invited participants. The quality of the responses will be measured by the number of surveys with missing or incomplete responses. For the interview portion, this will be measured by the number of survey items that the respondent chooses not to answer. The demographics of the respondents will be measured by the length of haul, i.e., long-haul, regional, or short-haul drivers and compared to data provided by FMCSA on the CMV driver population to assess the efficacy of each of the sampling strategies. It is important to note that FMCSA does not currently have a single source of CMV driver data for this information; rather, there are several sources that offer different estimates, so FMCSA will work to develop an estimate for comparing the survey results.

The dependent variables will be compared to determine if there is a main effect of survey administration method or the use of incentives or any interaction between the two. The results of will be used to develop a methodology for a larger information collection and provide FMCSA with a set of guidelines for conducting surveys of the CMV driving population.

Final Sample Sizes: The final sample size of a survey takes into account the various sources of variance and non-response. These vary according to the individual population being sampled and the survey method.

Mail-in: For each sampling frame, if one survey were mailed to commercial motor vehicle drivers, the desired response size would have to be augmented as follows:

200/.30 = 667 (for an estimated mail response rate of 30%. We plan to send a reminder postcard and a second set of survey forms to drivers who did not respond to the initial mailing to increase the overall response rates. Additionally, an incentive will be offered to half the survey recipients).

667/.90 = 742 (10% reduction in response rate to cover out-of-date mailing addresses for drivers in the company databases and language issues)

Driver Interviews: The estimate of the number of respondents must consider the greater homogeneity in the population that will be sampled than would be present for a simple, random sample. It is also important to consider the potential non-responses. Thus, the final sample size calculations for each location are as follows:

200/.80 = 250 (for an estimated driver intercept response rate of 80%)

250/.99 = 253 (1% reduction in responses due to language issues, e.g., drivers who are not proficient in the English language. The 1% was calculated as a percentage of the number of drivers who have been cited for a lack of English proficiency violation relative to the total number of drivers receiving a violation).

In summary, the final sample sizes for the two surveys according to the sampling method and sampling frame are:

Sampling Method	Respondent Universe	Sample Size	Expected Overall Response Rate	Expected Number of Responses
Mail-in	AAMVA	742	30%	200
	Industry partnership	742	30%	200
	TOTAL	1484	30%	400
Interviews	DMV Offices	253	80%	200
	Truck Stops	253	80%	200
	Weigh Stations	253	80%	200
	TOTAL	759	80%	600

A power analysis using the expected number of responses (400 for mail-in and 600 for interviews) indicated that a difference in response rates of 11% could be detected with 95% confidence and power 0.95. The expected difference in response rates of 50%, as currently hypothesized, can be detected with power almost 1.

Final Estimates: Final estimates will be presented in the form of percentages and distributions.

3. Describe methods to maximize response rate.

Half the respondents would be offered a $10 gift card for participating in the survey. This incentive is expected to increase the response rate. The survey would also be kept to a maximum of 15 minutes, and the survey items will be worded so that each question is clear and unambiguous.

Mail-in surveys traditionally have a lower response rate than driver interviews. Based on research-tested recommendations in the text, Mail and Internet Surveys: The Tailored Design Method², by Don Dillman, the steps taken to maximize the response rates include:

• Use respondent-friendly design – Our questionnaire designs were reviewed and modified by an in-house design expert. The final design incorporates prominent question numbers, bright labels, and clear instructions for guiding respondents through the questionnaire. Color was considered but not used for cost reasons.

• Pre-notice letter / replacement questionnaire – A pre-notice letter will be sent a week before the first questionnaire. The survey plan calls for a second set of questionnaires to be sent out to all respondents who do not return the first questionnaire after a period of two to three weeks.

• Thank you note / reminder – The survey plan will include a thank you / reminder postcard to be sent out one week to 10 days after the initial mailing.

• Real stamp on return envelope – A real stamp may be used on the return envelopes.

• Prominent disclosure that the return of the survey is mandatory technique cannot be used because the survey is voluntary.

• Questionnaire length – The questionnaire length was kept to the maximum of 15 minutes.

In addition, the following steps will also be taken to maximize response rates:

• The cover letter/introduction will be written to inform respondents of the importance of the information collection and the benefits of the study results to the truck driver.

• The questionnaire design will make it easy to respond with as many check-off answers as possible and clear, unambiguous wording of questions.

Non-response bias occurs when the observed value deviates from the population parameter due to differences between respondents and non-respondents. Non-response bias is likely to occur as a result of not obtaining high response levels (response rates less than 85 percent). Non-response bias analysis is conducted to determine whether or not the data are missing at random and to assess the potential magnitude of non-response bias. Comparisons of respondents and non-respondents across subgroups using available sample frame characteristics provide information about the presence of non-response bias. Some of the characteristics that will be compared include demographic information, such as gender, age, and the endorsements held. The potential magnitude of non-response bias can be estimated by taking the product of the non-response rate and the difference in values of a characteristic between respondents and non-respondents.

4. Describe tests of procedures or methods. 

The survey was pre-tested on a representative sample of 9 commercial motor vehicle drivers to refine and clarify questions and instructions only. The pilot test was conducted at two truck stops: the Flying J at Olive Branch, Mississippi, and a second in Maggie T’s in Batesville, Mississippi. Interviewers approached commercial motor vehicle drivers in the parking lot after the drivers had parked their rigs. All the drivers were read the survey in an interview format. Data collected during the pilot study consisted of time to complete the survey and feedback about the survey items. In the interview, the completion times were recorded from when the first question was read to after the last question was answered.

The data indicated that the survey took an average of 11 minutes to administer (the interview time ranged from 10 – 15 minutes). Participant feedback from the pilot study identified issues in the wording of a few of the survey items and/or the format of the responses. Feedback from drivers was incorporated into the questionnaire design, and revised survey was developed.

5. Provide the name and telephone number of individuals who were consulted on statistical aspects of the information collection and who will actually collect and/or analyze the information.

The FMCSA is sponsoring this information collection. The FMCSA contact is:

Dr. Mindy Shalaby, Economist

Office of Analysis, Research, and Technology

Telephone: (202) 493-0304

Email: Mindy. [email protected]

The Research and Innovative Technology Administration (RITA) Volpe Center and Bureau of Transportation Statistics (BTS) provided statistical expertise in the design, conduct and analysis of the information collection.

Dr. Promod Chandhok, Statistician

Telephone: (202) 366-2158

Email: [email protected]

The Volpe Center contacts are:

Dr. Michelle Yeh, Engineering Psychologist

Telephone: (617) 494-3459

Email: [email protected]

Dr. Scott Gabree, Engineering Psychologist

Telephone: (617) 494-2530

Email: [email protected]

1 From US DOT Volpe Center, Commercial Motor Vehicle Driver Economic Data Surveys: A Literature Review. Report in preparation.

2. Mail and Internet Surveys: The Tailored Design Method, Second Edition, Don A. Dillman, John Wiley & Sons, 2000.

6