Justification for the Renewal of OMB 21270646 Part B

B. Collections of Information Employing Statistical Methods

The proposed information collection will employ statistical methods to analyze the data collected from respondents. The following sections describe the procedures for respondent sampling and data tabulation.

B.1. Describe the potential respondent universe and any sampling or other respondent selection to be used.

The potential universe for these surveys includes all non-institutionalized adults in telephone households within the United States. With the exception of the Teen seat Belt demonstration project where NHTSA will survey teens (subject to institutional review board approval), adults are defined as persons aged 18 or older. The target population is 18 years and older because of the expense and limitations of getting institutional review board approval for administering surveys to drivers age 16 and 17 years olds. In addition because of the modest size of the national and State samples, the number of 16 and 17 year old that we would survey would be too small to be of any analytic value and justify the added expense of getting IRB approval.

This information collection will encompass multiple National samples and State samples. The sample will be distributed according to Table 4 followed by a more detailed description of each sample.

TABLE 4

Participants
	National Surveys	Demonstration Surveys	Total Participants
Annual	12,800	152,000	164,800
3 Year Total	38,400	456,000	494,400
Interview Hours
	National Surveys	Demonstration Surveys	Total Hours
Annual	2,133	25,333	27,467
3 Year Total	6,400	76,000	82,400

National Surveys

National samples will be required for both safety belt and impaired driving mobilization interventions. Two national safety belt mobilization and two national impaired driving mobilizations will be evaluated each year. For each mobilization, a pre-test (baseline) national sample and a post-test national sample of 1600 respondents will be surveyed (1200 respondents from a representative sample of the general population age 18 and older with an additional over-sampling (N=400) from media target group of males ages 18 to 34). Thus, the national sample for the proposed information collection would be:

(1600 respondents pre-mobilization + 1600 respondents post-mobilization) X

4 mobilizations (2 Alcohol Mobilizations + 2 safety Belt mobilization) X 3 Years = 38,400 respondents

Respondents for each national survey wave will be drawn from a national probability sample of households selected through a random digit dialing (RDD) sampling process¹. This number is sufficiently large to permit pre/post comparisons within reasonable bounds of sampling error as well as permit sub-sample analyses of some major demographic characteristics (e.g., age and sex). Screening criteria for age eligibility would be age 18 and older. Additional criteria will be used for the alcohol mobilization to restrict sampling to drivers, age 18 years old and older, who have consumed alcohol within the last year. The sample will be stratified according to four Census Regions: Northeast, Midwest, South, and West (see Table 6 for expected sample distribution and sampling error). Residents from all 50 States and the District of Columbia would be eligible for the sample. In total, 38,400 respondents will be used for the National Surveys over a three year period.

Table 6
2006 Projected Regional Census Population Age 18+ By U.S. Census Region, Completed Sample Size and Sample Error
Regions	States	Population	Proportion	Completed Sample N	*Sampling Error
US	50 States and DC	224,365,151	100.00%	1200	2.8%
Northeast	CT, MA, ME. NH, NJ, NY, PA, RI, VT	42,181,438	18.80%	225	6.5%
Midwest	IL, IN, IA, KS, MI, MN, MO, NE, ND, OH, SD, WI	49,887,732	22.24%	267	6.0%
South	AL, AR, DE, DC, FL, GA, KY, LA, MD, MS, NC, OK, SC, TN, TX, VA, WV	81,267,256	36.22%	435	4.7%
West	AK, AZ, CA, CO, HI, ID, MT, NV, NM, OR, UT, WA, WY	51,028,725	22.74%	273	5.9%
* Computed from the equation 1.96 X √p(q)/(n-1) where 1.96 is the z score at the 95% confidence level, p is the proportion of the sample displaying a particular characteristic (using the maximum value of the simple random sampling variance, or 50%), q equals (1-p), and n is the sample size. Source: File 2. Interim State Projections of Population for Five-Year Age Groups and Selected Age Groups by Sex: July, 1 2004 to 2030. U.S. Census Bureau, Population Division, Interim State Population Projections, 2005.

Demonstration Project Surveys

For each Demonstration project, a pre-test (baseline) sample and a post-test sample of up to 500 respondents per designated site sample will be surveyed. Sampling sizes will be sufficiently large so that sampling error is no greater than + 5 percentage points on questions administered to all respondents. Respondents will be selected from residential telephone exchanges covering the geographic area receiving the demonstration project intervention, using systematic procedures for sampling from exchanges and a random digit dialing (RDD) process for selecting numbers to call from sampled blocks of phone numbers. Age eligibility will depend on the nature of the intervention being evaluated. If the intervention is being directed at a youth population (i.e., under age 18), then the survey contractor shall first be required to undergo formal review and approval of methods by an Institutional Review Board certified by the Department of Health and Human Services.

Under this category, NHTSA is planning surveys for the following projects (described in detail in Section A.12.):

Table 7

	Annual	3 Year Total
Click It or Ticket: The Next Generation	16,000	48,000
Teen Seat Belt Demonstration Project	40,000	120,000
Seat Belt Demonstration Project in Rural Areas	24,000	72,000
Combined Alcohol and Seat Belt Enforcement Demonstration Projects	24,000	72,000
High Visibility Alcohol Enforcement Demonstration Projects	24,000	72,000
High Risk Population Alcohol Demonstration Projects	24,000	72,000

A total of 456,000 respondents will be used for the demonstration project surveys over a three-year period.

Total Sampling Needs

Overall, the total sample needs are 164,800 respondents annually, which would be 494,400 respondents over a three-year period. Since all the surveys are estimated to be 10 minutes in length, the estimated annual time is 27,467 hours interviewing, which would be or 82,400 hours over a three-year period.

Sampling Error

The confidence interval for sample estimates of population proportions, using simple random sampling without replacement, is calculated by the following formula:

Where:

se (x) = the standard error of the sample estimate for a proportion

p = some proportion of the sample displaying a certain

characteristic or attribute

q = (1 ‑ p)

n = the size of the sample

z = the standardized normal variable, given a specified

confidence level (1.96 for samples of this size)

To test whether or not a difference between two sample proportions is statistically significant, a rather simple calculation can be made. The maximum expected sampling error (i.e., confidence interval in the previous formula) of the first sample is designated s1 and the maximum expected sampling error of the second sample is s2. The sampling error of the difference between these estimates is sd and is calculated as:

• For comparison of two samples of 500 each, a difference would have to exceed 6.2 percentage points to be statistically significant (with the conservative estimate of p=q).

• For comparison of two samples of 1200 each, a difference would have to exceed 4.0 percentage points to be statistically significant (with the conservative estimate of p=q).

• For comparison of two samples of 1600 each, a difference would have to exceed 3.5 percentage points (with the conservative estimate of p=q).

We believe that this is sufficient for our purpose to assess the impact of the mobilizations

B.2. Describe the procedures for the collection of information.

The proposed surveys will be administered using a pre-/post-test design to examine the changes that occur as a result of specific safety belt and impaired driving interventions. The National and State mobilization surveys will be conducted on a schedule corresponding with the fixed annual dates for the national mobilizations. The demonstration project surveys will be conducted on a more variable schedule that will depend on the timing and sequencing of the components of each demonstration project.

National Mobilization Surveys

At the national level, data collected from random samples of 1600 people before and after the mobilizations will be compared to examine changes in awareness, attitudes and self-reported behavior.

The proposed national survey samples are based on a modified stratified random digit dialing method, using a geographically stratified RDD sample rather than a single-stage/RDD sample. There are several important advantages to using a geographically stratified base for the RDD sample: (1) it draws the sample proportionate to the geographic distribution of the target population rather than the geographic distribution of telephone households, which is vital to constructing unbiased population estimates from telephone surveys; (2) it allows greater geographic stratification of the sample to control for known geographic differences in non-response rates; and (3) it facilitates the use of Census estimates of population characteristics to weight the computed sample to correct for other forms of sampling bias

The initial stage of the sample construction process requires the development of a national area probability sample based upon the distribution of the target population for this study, i.e. the non-institutionalized population age 18 and older in the United States. The estimated distribution of the population age 18 and older was calculated on the basis of projected 2006 data from the U.S. Census Bureau, (Table 6 shows the distribution of the population age 18 and older according to the census regions, the proportionate sample size for each region, and the sampling error).

Once the sample has been geographically stratified with sample allocation proportionate to population distribution, a sample of assigned telephone banks will be randomly selected from an enumeration of the Working Residential Hundreds Blocks of the active telephone exchanges within the region. The Working Hundreds Blocks are defined as each block of 100 potential telephone numbers within an exchange that includes 3 or more residential listings. (Exchanges with one or two listings are excluded because in most cases such listings represent errors in the published listings.).

In the third stage sample, a two-digit number is randomly generated by computer for each Working Residential Hundreds Block selected in the second stage sample. This third stage sampling technique is known as random digit dialing (RDD). Every telephone number within the Hundreds Block has an equal probability of selection, regardless of whether it is listed or unlisted. The use of RDD sampling eliminates the otherwise serious problem of unlisted telephone numbers.

The sample construction described above yields a population-based, random digit dialing sample of telephone numbers. The systematic dialing of those numbers to obtain a residential contact should yield a random sample of telephone households. During the fourth stage of sampling, a random selection procedure will be used to select one designated respondent for each household sampled. The “most recent/next birthday method” will be used for within household selection among multiple eligible respondents. Salmon and Nichols (1983²) proposed the birthday selection method as a less obtrusive method of selection than the traditional grid selections of Kish, et al. In theory, birthday selection methods represent true random selection (Lavrakas, 1987³). Empirical studies indicate that the birthday method produces shorter interviews with higher response rates than grid selection (Tarnai, Rosa and Scott, 1987⁴).

Upon contacting the household, interviewers will briefly state the purpose of their call (including noting the anonymity of the interview), and then request to speak to the person in the household within the eligible age range who has had the most recent birthday, or will have the next birthday. The CATI system will randomly rotate whether the interviewer asks for the most recent or next birthday. If the person who answered the phone is the selected respondent, then the interviewer will proceed with the interview. If the selected respondent is someone else who then comes to the phone, then the interviewer will again introduce the survey (with anonymity statement) and proceed with the interview. If the selected respondent is not available, then the interviewer will arrange a callback.

Demonstration Surveys

The major differences between national and State Click It or Ticket Next Generation sample selection procedures will be sample size and the absence in most cases of Stage 1 distribution of sample by geographic stratification. Otherwise, sampling procedures will mirror the same procedures described above for selecting the national samples (i.e., Stages 2-4). The national and State samples will be selected independently. As indicated in Section B1, NHTSA may determine that some level of stratification is desirable for analytic purposes. Therefore, sample generation for one or more States could include some form of Stage 1 distribution of sample by geographic stratification.

Other demonstration projects will typically be directed towards a community, a county or a media market composed of multiple counties. The residential telephone exchanges covering the geographic area undergoing the intervention will be determined, and a systematic procedure for randomly selecting telephone numbers to call will be implemented. Demonstration project surveys may require more screening criteria than age as interventions may be directed at very specific subgroups within the community. In-house selection methods will be conducted that obtain scientifically valid random samples.

Data Collection Procedures across Samples

Data collection will be conducted by trained interviewers working in telephone research centers that utilize a computer-assisted telephone interviewing (CATI) network. The CATI network will have capability for silently monitoring the performance of interviewers. Monitoring will be conducted by supervisory staff during all interview shifts to determine the quality of interviewer’s performance in terms of:

1. Initial contact and recruitment procedures;

2. Reading the questions, fully and completely as written;

3. Reading response categories, fully and completely, (or not reading them) according to the study specifications;

4. Whether or not open-ended questions are properly probed;

5. Whether or not the interviewer enters the correct code, number, or verbatim response to the question;

6. Whether or not ambiguous or confused responses are clarified

7. How well questions from the respondent are handled without alienating the respondent or biasing his/her response;

8. Avoiding bias by either comments or vocal inflection;

9. Ability to persuade wavering, disinterested or hostile respondents to continue the interview; and

10. General professional conduct throughout the interview.

Initial telephone contact will be attempted during the hours of the day and days of the week that have the greatest probability of respondent contact. This means that the primary interviewing period will be conducted between 5:30 p.m. and 10:00 p.m. on weekdays; between 9:00 a.m. and 10:00 p.m. on Saturdays; and between 10:00 a.m. and 10:00 p.m. on Sundays. If the interview cannot be conducted at the time of initial contact, the interviewer will reschedule the interview at a time convenient to the respondent. Although interviews will be conducted on evenings and weekends whenever possible, daytime interviews will be scheduled whenever necessary.

Statistical Analysis

Sample selection for all surveys will follow as closely as possible simple random sampling, with some stratification occurring for the national sample. NHTSA will weight the national and demonstration samples by the likelihood of selection (sex, number of telephone lines and number of eligible adults in the household) and then by age and sex to most recent Census estimates for the specified geographic area. However, NHTSA will not weight the national and demonstration samples for the Alcohol mobilizations because the sample is restricted to drivers age 18 and older who have consumed alcohol within the last year, and there are no census parameters on drivers that drink to which we could weight by age and sex. In cases where over-sampling occurs, the weighting system will compensate for the over-sampling.

Chi square statistics will be applied to final data to compare results from survey waves. . Specifically, statistical tests will be used to determine id f there are statistically significant differences between pre and post waves. Additional statistics may be calculated if NHTSA sees a need for more refined analyses and special statistical software (i.e. SPSS, SAS) will be used if any data are collected using a complex sample design.

B.3. Describe methods to maximize response rates.

One of the steps that NHTSA has considered in order to try to increase response rates for the national mobilization is extending the standard two-week field period. However, a two-week field period is used for these mobilizations because of the constraints involved in coordinating data collection with several States. Specifically, in order to avoid contamination from State highway safety activities, NHTSA confines data collection to a two-week period prior to the mobilization and a two-week period after the mobilization.

The national mobilizations and demonstration projects will include five call attempts and seven callbacks during the field periods. However, the limited field periods will require that the surveys place particular emphasis on contact scripts and the training/monitoring of interviewers. The initial contact script has been carefully developed and refined to be persuasive and appealing to the respondents. The interviewing will be conducted only by thoroughly trained and experienced interviewers who are highly motivated and carefully monitored. All interviewers will have had training on how to overcome initial reluctance, disinterest or hostility during the contact phase of the interview. There will be maintenance and regular review of field outcome data in the sample reporting file, derived from both the sample control and CATI files, so that patterns and problems in both response rate and production rates can be detected and analyzed. Periodic meetings will be held with the interviewing and field supervisory staff and the study management staff to discuss problems with contact and interviewing procedures and to share methods of successful persuasion and conversion. Within the field period of the surveys, five call attempts and seven call backs will be used to maximize response rates.

The methods NHTSA uses to increase response rates, which we are now seeking to renew, achieved American Association for Public Opinion Research (AAPOR) response rates ranging from 40% to 45%. This is above the industry standards, which are estimated to be closer to 30%. We will attempt to determine how representative our samples are with respect to the target population by comparing demographic breakout of our samples with demographic profiles from census data. Furthermore, NHTSA has observed strong consistency between the results of its telephone surveys and both known activity in the field as well as concurrent research utilizing self-report or other data collection techniques. The following section is included to provide further evidence of the validity of these surveys.

The “Click It or Ticket” (formerly “Buckle Up America”) surveys have been conducted by NHTSA since 1998. Pre/post survey waves have shown questions in the data collection instrument to be sensitive to the national enforcement mobilizations, as illustrated in Figures 1 and 2. Moreover, the Air Bag and Seat Belt Safety Campaign (AB&SBSC) also conducted telephone surveys during the early years of the safety belt mobilizations. Their results were consistent with the NHTSA results for similar questions (e.g., perceived risk of being ticketed.

The core self-report behavioral and demographic questions on the Click It or Ticket (CIOT) Surveys are also on NHTSA’s Motor Vehicle Occupant Safety Survey (MVOSS), which has used the same basic methodology as the CIOT surveys except for the addition of over-sampling younger persons. The MVOSS formerly was administered approximately the same time of year as NHTSA’s national probability observation survey of safety belt use (the schedule of the observation survey was drastically changed a few years ago). As shown in Table 8, there has been a strong correspondence between the telephone and observational data. The self-report surveys, through the most recent versions, have continually shown the same patterns detected in the national observation surveys (lower belt usage among males, younger persons, pickup truck drivers, etc.).

F igure 1. Seen or Heard Special Enforcement Efforts In Past 30 Days

Q14. In the past 30 days, have you seen or heard of any special effort by police to ticket drivers in your community for seat belt violations?

Base: Total adults (Unweighted N’s range from 1,000 to 1,212)

*Differences between Pre and Post measurement are significant at the .05 level

The sequencing of the Summer 1998 surveys did not follow a pre/post design. Rather, the first survey wave occurred during the period of the intervention, whereas the second survey wave occurred a couple of months after the intervention period.

The larger increases in the more recent years reflect the introduction of paid media.

Figure 2. Police are Writing More Tickets for Seat Belt Violations Now: Strongly/Somewhat Agree

Q13f. Police in my community are writing more seat belt tickets now than they were a few months ago.

Base: Total adults (Unweighted N’s range from 1,000 to 1,212)

*Differences between Pre and Post measurement are significant at the .05 level

The larger increases in the more recent years reflect the introduction of paid media.

TABLE 8 Revised Reported Seat Belt Use Compared To Observed Use By Drivers
	1998 MVOSS (Telephone Survey) "All Of The Time"	Revised 1998 MVOSS (Telephone Survey) "All Of The Time" (Excludes past day or week non-users)	1998 NOPUS (Observation Survey) Drivers
Total Drivers	79.2%	71.4%	69.6%
Males	74.1%	65.4%	64.3%
Females	84.2%	77.2%	77.7%
Blacks	75.2%	69.5%	67.5%
Whites	78.9%	70.9%	70.3%
Age 16-24	76.0%	63.9%	58.4%
Age 25-69	79.1%	72.2%	70.5%
Age 70+	85.0%	76.7%	76.4%
Passenger Cars	82.3%	74.3%	73.8%
Pickup Trucks	64.7%	57.6%	52.8%
Urban	79.7%	71.8%	74.5%
Suburban	79.7%	72.2%	67.6%
Rural	77.4%	68.9%	67.0%

Source: NHTSA Motor Vehicle Occupant Safety Survey: Volume 2 Seat Belt Report, March, 2000

B.4. Describe any tests of procedures or methods to be undertaken.

The proposed mobilization surveys are a continuation of safety belt and impaired driving mobilization surveys conducted in previous years. As such, they will utilize questionnaires nearly identical to those utilized previously, and follow methods that have been previously implemented and found successful.

B.5. Provide the name and telephone number of individuals consulted on statistical aspects of the design

The following individuals consulted on statistical aspects of the study design:

Richard Compton, PhD

Director, Office of Behavioral Safety Research

DOT/National Highway Safety Administration

1200 New Jersey Ave, SE

Washington, DC 20590

(202) 366-2699

Maria Vegega, PhD

Chief, Behavioral Research Division

Office of Behavioral Safety Research

DOT/National Highway Safety Administration

1200 New Jersey Ave, SE

Washington, DC 20590

(202) 366-2668

Linda Cosgrove, PhD

Chief, Injury Prevention Research Division

Office of Behavioral Safety Research

DOT/National Highway Safety Administration

1200 New Jersey Ave, SE

Washington, DC 20590

(202) 366-5592

Alan Block, MA

Office of Behavioral Safety Research

DOT/National Highway Safety Administration

1200 New Jersey Ave, SE

Washington, DC 20590

(202) 366-6401

John Siegler, PhD

Office of Behavioral Safety Research

DOT/National Highway Safety Administration

1200 New Jersey Ave, SE

Washington, DC 20590

(202) 366-3976

1 In 2007, our office conducted an experimental cell phone only study at the behest of OMB that compared a RDD sample with a cell phone only sample. The results of this study led to a decision not introduce cell phone sample into the MVOSS because of the design effects. We are continuing to study the cell phone issue, but believe the decision made with respect to MVOSS is still applicable for the next few years.

2 Salmon, C. and Nichols, J. The Next-Birthday Method of Respondent Selection. Public Opinion Quarterly, 1983, Vol. 47, pp. 270-276.

3 Lavrakas, P. Telephone Survey Methods: Sampling, Selection and Supervision. Beverly Hills: Sage Publications, 1987.

4 Tarnai, J., Rosa, E. and Scott, L. An Empirical Comparison of the Kish and the Most Recent Birthday Method for Selecting a Random Household Respondent in Telephone Surveys. Presented at the Annual Meeting of the American Association for Public Opinion Research. Hershey, PA, 1987.

12