10, 09-03 Hit-Run Pedestrain Study-Part B

Supporting Statement for Paperwork Reduction Act Submission (Part B)

Office of Management and Budget (OMB) Control Number 2127-XXXX

An In-Depth Examination of Pedestrian-Involved Hit-and-Run Crashes

September 3, 2010

A. JUSTIFICATION. Please see Part A.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

1. Respondent universe

The respondent universe is the same for the pilot as it is for the full survey.

The respondent universe theoretically consists of all drivers in the United States who hit a pedestrian. This includes all collisions with pedestrians whether the driver did or did not leave the scene of the collision and whether anyone was injured or killed in the crash. However, the resources available for this study, and the difficulty of obtaining a sample, do not permit the most desired approach—a stratified random sample of the United States. There does not exist a list of all drivers involved in a hit-run collision from which we could generate a sample. Instead, the goal is to select geographic areas for study that represent a large proportion of hit-and-run collisions in the United States. Drivers in these areas will be the respondent universe. They would only represent drivers who hit a pedestrian in those geographic areas, not all drivers who hit a pedestrian. Within these areas, NHTSA will attempt to identify and contact drivers for participation in the survey. Comparisons and assessments of the characteristics of the drivers who were identified and included in the survey with other drivers to assess potential biases in the sample will be made if possible. Based on the past experience of the survey team with sensitive surveys, the expected response rate is approximately 10 percent. We will compare the drivers who respond with other drivers involved in hit-run collisions with respect to available information (e.g., age, gender).

The study will select 15 local areas to be evaluated as potential sites for intensive data acquisition and analysis. Ten areas will be selected from the 15 for detailed statistical analyses and for specifying the sample of drivers for a telephone survey and/or for focus groups. The focus of the survey will be on counties because:

• Counties represent a natural bureaucratic unit for which a great deal of data exist.

• There is sufficient variation within most counties of factors that are likely to affect the occurrence of hit-and-run crashes.

• Analyses can still be done on specific jurisdictions (e.g., cities, unincorporated areas) within counties.

If necessary, and if the total incidence of pedestrian hit-and-run crashes is insufficient from single counties alone to guarantee a large enough sample size for our driver survey, it may be necessary to include neighboring counties in a study area. The need for a sufficiently large volume of data will be balanced against the ease of obtaining the data.

2. Sampling methods

The study will employ a simple multi-stage method for selecting a sample of respondents. Because serious crashes are rare relative to the quantity of driving experienced by the typical driver, we must employ a sampling method that allows for the clustering of respondents to improve the ability of the researchers to gain access to detailed police and court records on each crash. The clusters will be counties that are selected based on the total number of fatal pedestrian-involved hit-and-run collisions recorded in 10 years of FARS data and the proportion of these crashes relative to total fatal pedestrian-involved crashes. To ensure that there are enough individuals to sample, the 20 counties with the largest absolute total number of fatal pedestrian-involved hit-and-run crashes will be identified and, of those, the 15 with the highest ratio of fatal hit-and-run crashes over the number of pedestrian-involved crashes for the county. The researchers will contact police and court data custodians to obtain detailed information on pedestrian-involved hit-and-run crashes for the 10 year period preceding the data collection effort. The data will include names and, when available, contact information for drivers. At this time, the researchers will determine the number of drivers with contact information that are available for inclusion in the sample frame.

 

Within the 10 clusters, the sample frame will consist of two strata of drivers (hit/no-run and hit/run/apprehended) identified through obtaining crash and judicial records from traffic and criminal courts in the individual jurisdictions. The records will be used to identify, select, and scan for the names of drivers implicated in pedestrian-involved crashes. Any drivers that faced criminal proceedings must have received their judgment. Having identified the two groups of drivers, the team will conduct telephone interviews of subjects in the two groups, for a total of up to 200 completed interviews in the pilot and up to 700 completed interviews in the full survey. This sample size will provide adequate statistical power for the planned analyses.

Assuming that response rates are similar among the ten focus areas, we will obtain a proportional number of respondents according to the size of the sampling frame available in each focus area. It is not anticipated that there will be a need to further stratify below the focus area level. The aim is to have the results be generalizable to the population of drivers in the selected jurisdiction.

A study of this sort, surveying drivers who have been involved in a pedestrian hit-run collision, has never been accomplished before. Given the difficulties in acquiring a sample of such drivers to survey, it will be impossible to randomly select from a specified universe. Therefore, the information gained will be presented as an example of issues involving drivers in pedestrian hit-run collisions. To strength the results somewhat, we will be able to compare information from the collision reports for drivers who did and who did not participate in the survey. This will determine whether drivers participating in the survey are similar or different with respect to demographics and time and circumstances of the collision.

The number of completed interviews will depend on the team’s ability to identify and recruit subjects from the target population. Our aim is to obtain interviews from 200 drivers in the pilot and an additional 700 drivers in the full survey. However, given the uncertainty in obtaining contact information and cooperation in this unique study, we cannot guarantee the overall goal of 900. We have determined that even with fewer than 900 drivers, useful information will still be obtained although with less precision. The team will use established procedures to maximize the survey response rate and accuracy of data obtained, including developing training materials and training survey staff and supervisors; multiple attempts to obtain data from interviewees; and controlling for sampling and non-sampling error. Reverse lookup will be used to find phone contacts for individuals where it is not available in the record. A 20-percent success rate at reaching the contacts is anticipated. The research team will finalize the approach after collection of the criminal and traffic records from the 10 focus areas and determination of the level of contact information.

2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection

* Estimation procedure

* Degree of accuracy needed for the purpose described in the justification

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

1. Sample selection

Two of the ten local areas will be used in the pilot study. Interviews will be conducted until 100 complete interviews in each of the areas are obtained. For the full study, the remaining 8 areas will be used, in order to obtain an additional 700 complete interviews.

Because of human subject and legal considerations, we are only able to include individuals in our sample who have completed the adjudication process. Within this population for each site we will randomly select individuals to contact for interviewing from the sampling frame until the sampling quota is met. Descriptive statistics (mean and standard deviation) will be calculated for the group of interviewed individuals.

We intend to calculate descriptive statistics for each of the 10 sites. When the process is complete, we will test whether the distributions of key variables differ from each other with respect to mean and standard deviation. If there is not an overall statistically significant difference across sites, we will combine the sites for calculation of confidence intervals. If we are able to achieve the target of 900 interviews across the 10 sites, the resulting 95% confidence interval will be about +- 3 percentage points (for an observed proportion of 50%). If the samples differ significantly, we will calculate confidence intervals separately. With about 90 completed interviews for an individual site we will have a 95% confidence interval of about +- 10 percentage points. While this is not ideal, it should be noted that this is the only data collected to date for this population; an estimate with a confidence interval of +- 10 should provide valuable, if approximate, estimates of variables assessed in the survey. It is clear that our results cannot be generalized in a strict statistical sense beyond the individual sites where our survey will be conducted. However, the sites chosen are in fact among the largest population areas in the country, with the largest number of hit-run collisions, and, therefore, encompass a large proportion of the overall hit-run problem. In addition, we intend to compare features of our sample of drivers (and their corresponding collisions) to characteristics of the equivalently defined national population of drivers and collisions. From this comparison we at least will know if our sample is similar to the national population of drivers in pedestrian-involved hit and run crashes.

2. Data analysis

The research team will begin the survey analysis of the pilot by performing descriptive statistics (tabulations and cross-tabulations) using SAS and/or STATA statistical software, accounting for the sampling methodology. Descriptive statistics will include profiles of the two groups of drivers surveyed, with characteristics such as mean age, gender breakdown, injury severity, years of licensure, and previous crash history. Of particular interest will be the overall response rate per strata of drivers, by focus area, as well as the response rate for sensitive questions. This would allow identifying whether any of the questions are not being answered and/or appear to provide invalid or unreliable answers. Based upon these analyses and overall results, the value of obtaining additional complete interviews will be accessed. Recommendations regarding revisions to the survey instrument will be developed based on the pilot results, and will be revised for additional data collection.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

1. Interviewing plan

Telephone interviews for the pilot study are scheduled to occur within a 3-month period from October 2010 through December 2010.

All telephone surveys will be conducted in a state-of-the-art CATI laboratory located in the E&W office in San Francisco, with an independently networked CATI server system. All systems are password protected as well as internally firewalled to maximize security and guarantee the highest degree of confidentiality for respondents and for the collected data. All telephone interviewing staff are trained on both applications to ensure immediate troubleshooting and an error-free system operation.

The finalized survey instruments will be programmed into Sensus programming code. Skip patterns and qualifying criteria are built into the program to operate automatically, allowing the interviewer to focus mainly on interviewing. Ultimately, the integrated scripted protocols will direct respondents electronically through the telephone survey process.

At least 15 attempts will be made to reach households with repeated busy tones, answering machine pickup, or unanswered calls. Each new attempt occurs on a different day and/or at a different hour than any previous attempt. The CATI software automatically handles callback scheduling, but also allows interviewers to schedule selected callbacks for specific dates and times. Also, interviewers may attach high priority to certain cases, such as hard-to-reach respondents, to bump the cues for these cases ahead of other appointments that may be scheduled for the same time. When a busy signal is reached, CATI automatically cues that case again after 20 elapsed minutes. These procedures minimize cases of “non-response with eligibility unknown.”

The proposed dialing timeframe will be Mondays through Fridays from 3 pm to 9 pm and Saturdays and Sundays from 12 pm to 5 pm at the respondent’s respective time zone. At the respondent’s request, E&W will schedule callbacks outside this timeframe. In addition to scripted messages, a toll-free number will be left for participants to call to schedule interviews. The toll-free voicemail message script will be developed by the study team and will identify the study by acronym only.

Interviewers are supervised by trained supervisors with a target ratio of 1:10 to 1:20 supervisors per interviewers. The ratio is adjusted according to the complexity of the survey instrument, the experience of the interviewers, and the length of time the instrument has been operational, with additional supervisory hours dedicated at the beginning of a study when most problems will become evident. In addition, all telephone interviewers are monitored in real time and are evaluated at least once for every 4-hour shift using a quality control assessment tool programmed and running concurrent with the CATI software. Any protocol discrepancy or other need for improvement is continuously communicated to every telephone interviewer, and only interviewers who have successfully mastered the survey research protocol and survey techniques are permitted to conduct telephone interviews for any research project.

In addition, all survey research staff have received a certificate of participation from a multi-mode online course on Human Subject Assurance Training provided by the Office of Human Research Protection (OHRP) of the HHS.

All interviews are conducted from the E&W CATI laboratory located in San Francisco. Client staff representatives are welcome to visit the E&W offices at any time to observe interviewing in person.

2. Accuracy and reliability

The measures taken for accuracy and reliability will be the same for the pilot data as it will be any additional data received from the full survey.

Given that few, if any, studies have attempted to find and survey the population of drivers implicated in pedestrian-involved crashes, estimating the response rate is difficult. To determine level of effort, an estimated response rate of about 10 percent is used. The survey team has extensive experience in epidemiology follow-up studies and should be able to apply this experience to attempt to obtain contact information for names identified in court or collision records so that we can begin with as large a pool of contacts as possible.

To deal with non-response, established methods will be used to compare respondents and non-respondents using crash data obtained from police and court records and to adjust data obtained from respondents. The characteristics of responders to non-responders will be compared to identify under sampled strata. If necessary, multiple imputations will be used to weigh responses and avoid systematic bias in the analysis.

To test the level of accuracy for respondents, answers to the survey will be compared to information obtained in police or court records, such as the time and location of the collision. This will help us estimate response error and response bias for sensitive questions. Response rates are another measure of the extent to which a data set accurately reflects the characteristics and responses of a given population. Groves (1989) suggests that non-response rates are actually a composite of two factors whose underlying causes may be substantially different—non-contacts and non-interviews (i.e., refusals). The study team will assess both of these components.

1. No Contacts/Non-Response

In accordance with OMB requirements, the survey team proposes to conduct a non-response bias study once the primary data collection has been completed. In a recent work, Groves¹ reports that there is no consistent relationship between response rates and non-response bias. As such, a lower response rate may not necessarily cause or result in non-response bias. However, it does not imply that this finding will be true in all circumstances. The study team, therefore, plans to undertake a separate non-response bias study to examine the non-response patterns and to assess the potential for non-response bias in this survey.

The non-response bias analysis will involve conducting an analysis of demographic characteristics using census or similar external data sources. The estimates based on non-respondents will also be compared, to the extent possible, to estimates based on respondents or those based on external data. If necessary, the study team may, for the purpose of better understanding the non-response patterns, investigate the suitability of examining select subgroups within the group of respondents for the main study. For example, the team may examine: (1) those that are “easiest to reach and interview” as measured by records of calls in our CATI system; and (2) those that are “more difficult to reach” (require more callbacks) compared to group (1) above. The goal of the non-response bias study will be to detect whether significant and policy-relevant differences exist between the survey estimates for respondents and the non-respondents.

2. Refusals

Refusals appear to be increasing and tend to account for a major proportion of non-responders with the potential for non-response bias. Although the reasons are unclear, the immense expansion of telemarketing activities, a possible tendency toward greater resistance to perceived intrusions into the privacy of one’s home, and the increasing telephone saturation among certain market segments of the population may all contribute. The survey team will make every attempt to keep the refusal rate to a minimum.

An interaction with a respondent will be coded as a refusal if an interviewer has encountered two “soft refusals” after a description of the study, (e.g. “I am really not interested.” or “I’m too busy to talk to you now.”) or a hard refusal (e.g. “I’m not interested; don’t call again.” or “No, I don’t want to do it!”). Any case where a person hangs up before the interviewer can complete the introduction will be called back at least once.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The survey has already been tested on several individuals who work with the survey group, several of which will actually work on the study. The purpose of these administrations to survey group staff was to test the survey instrument on a wider sample and familiarize interviewers with the survey instrument. Adjustments were made to the survey based on the results.

An additional test of the full survey form is this proposed pilot test to be conducted with a target of 200 completed surveys. The goal of the pilot test is to assess the workability of newly added or modified question wordings, and to check the flow and general understandability of survey questions. Additional adjustments will be made to the survey form as necessary based on the results.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Project Leads for this information collection:

Charlene Doyle Linda C. Sharpe

Evaluation Division Senior Associate

Office of Regulatory Analysis and Evaluation Booz Allen Hamilton Inc.

National Center for Statistics and Analysis 22 Batterymarch Street

National Highway Traffic Safety Administration Boston, MA 02109

US Department of Transportation (617) 428-4445 (voice)

NVS-431 / #W53-469 (617) 428-4410 (fax)

1200 New Jersey Avenue, S.E. [email protected]

Washington, DC 20590

(202) 366-1276 (voice)

(202) 366-3189(fax)

[email protected]

1 Groves, Robert M. 2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70: 646-675.

7