RAPT_ICR_PartB_for_OMB

Information Collection Request Supporting Statement: Section B

Hazard Perception and Distracted Driving Training Intervention for Teens Evaluation

The National Highway Traffic Safety Administration (NHTSA) of the U.S. Department of Transportation (USDOT) is seeking approval from the Office of Management and Budget (OMB) to collect information from newly-licensed teen drivers to determine (1) their eligibility to participate in a study to evaluate a training intervention for teens to improve driving safety, called Risk Awareness and Perception Training (RAPT), (2) whether, during the first six months of driving, new drivers who complete the training have fewer crashes or traffic violations on their driving records than comparison group members who receive placebo training, (3) when they do crash, whether there is a difference in severity and at-fault between drivers taking the training versus those not taking the training, and (4) whether there is an interaction between sex and training group as measured by exposure, crashes, or crash characteristics.

Ten thousand teen volunteers will be recruited for the study within two weeks of passing the on-road driving test required to obtain their first license. A roughly equal distribution of males and females will be sought within each age group (16-, 17-, 18-, and 19-year-old drivers). Within each sex and age group, drivers will be randomly assigned to either the treatment (RAPT) or the control group (placebo—an educational video about vehicle maintenance). The total sample size of 10,000 teens should have substantial power to detect meaningful differences in driving performance. In the previous NHTSA study “Evaluation of the safety benefits of the risk awareness and perception training program for novice teen drivers” (Report No. DOT HS 812 235), reductions among males in crash rates of 24% were found with approximately 5,000 participants. On the other hand, no significant results were found for females. (There was an increase in crash rate of 11%, but it was not statistically significant.) With our planned number of study participants of approximately 10,000, the sample power to detect a reduction in crash rates of 24% (i.e., that detected in the previous study) at a significance level of 5% (alpha = 0.05) will be 0.94 (beta = 0.06).

The current plan is for the final technical report to be published in late 2021. The technical report will provide summary statistics and tables, as well as the results of statistical analysis of the information, but it will not include any personal information.

B.1. Describe the potential respondent universe and any sampling or other respondent selection to be used.

The respondent universe is defined as teens of 16, 17, 18, or 19 years of age who have recently (i.e., within two weeks) obtained their first drivers’ licenses. The researchers plan to sample from this universe in two jurisdictions at local DMVs from a captive audience, i.e., immediately following the obtaining of their license. To ensure that our sample will be sufficiently large, we plan to recruit for at least nine months in two jurisdictions. The primary jurisdiction from which we plan to collect data is Washington State. Its estimated population of teens (15-19) in 2015 was 440,265, while the number of licensed teens (16-19) in 2013 was 199,118. We also plan to recruit participants from a secondary jurisdiction to increase the study’s external validity; this jurisdiction is Michigan. In 2016, the estimated teen population (15-19) in Michigan was 668,075, while the estimated number of licensed teens (16-19) was 471,135.

While the participants will be randomly assigned to treatment versus control groups, participants will be accepted on a first-come, first-served basis and will not be sampled for study inclusion. There is no reason to believe that the study participants will be substantially different from all eligible participants nationwide.

The total sample size of 10,000 teens should have substantial power to detect meaningful differences in driving performance. In the previous NHTSA study “Evaluation of the safety benefits of the risk awareness and perception training program for novice teen drivers” (Report No. DOT HS 812 235), reductions among males in crash rates of 24% were found with approximately 5,000 participants. On the other hand, no significant results were found for females. (There was an increase in crash rate of 11%, but it was not statistically significant.) With our planned number of study participants of approximately 10,000, the sample power to detect a reduction in crash rates of 24% (i.e., that detected in the previous study) at a significance level of 5% (alpha = 0.05) will be 0.94 (beta = 0.06).¹ At beta = 0.2, we have the sample power to detect a reduction in crash rates of 20%.

B.2. Describe the procedures for the collection of information.

After obtaining consent, teens will be asked questions to determine eligibility for the study as well as for group assignment. Teens who agree to participate in the study will initially be asked whether they took driver education, and, if so, if it included hazard perception training. Teens who have already completed a hazard perception training program will be excluded from (ineligible for) the study.

The data collection calls for collecting pre-training measures of hazard recognition, hazard recognition testing immediately following the RAPT or placebo training, and again after six months. Detailed exposure data will also be obtained by inviting a subsample of 2,000 teens from the primary jurisdiction to complete a log of driving on each day of the past week indicating time driven, miles driven, and number of trips taken. This subsample of teens will be distributed across each of the six months during the follow-up period so that robust data for each month will be available.

At the six-month follow-up, teens from the primary jurisdiction will be asked questions about any crashes or traffic tickets during their first six months of driving. Crash and violation data will be also be obtained from driver records. Both data collection methods for crash data are needed because not all crashes are reported to the police, so self-reports provide a more complete count of all crashes. Police-reported crash data, however, may provide more objective and accurate details than self-reported crash data, which are subject to poor recall and cognitive biases. Driver records data (e.g., crashes, crash type, and citations) will be obtained from DMVs in the primary and secondary jurisdictions based on data sharing agreements. Collision abstracts will also be reviewed to obtain at-fault information. Data merging protocols using unique identifiers and indexing will be used to merge and append all the data from each respondent.

B.3. Describe methods to maximize response rates.

While participation in this study is voluntary, the research team will rely on using a proven method of distributing incentives to increase participation rates and reduce attrition. Respondents will be invited to participate, and $2 (two one-dollar bills) will be included in each envelope along with the invite letter. The reason for doing this is that offering a small monetary incentive before the decision to participate garners good will and motivates teens to participate. This will increase the chances of successfully recruiting the required number of teen volunteers in this study. This method is consistent with Dillman’s tailored design method² and has been successfully applied in other studies by the contractor (notably, in an evaluation project of driver education conducted in Oregon and Manitoba³).

Once recruited, study participants will be offered $5 as compensation for completing the training or placebo training protocols. A five-dollar bill will be mailed once they have completed the protocol. Study participants who complete the six-month follow-up test will receive a further $10. Similarly, those who are invited to complete the drive log and complete it will receive $10. This will also be mailed to them once they have completed this follow-up test and/or the drive log.

Our previous experience working with large samples of teens indicates that using this method of dispensing small amounts merely to invite teens to participate, in combination with higher, yet still relatively small, amounts to compensate for actual participation, works well to recruit participants from this population.

B.4. Describe any tests of procedures or methods to be undertaken.

Table 1 shows some of the main research questions of interest and the analysis approaches the research team will use to formally test each question. For example, to determine whether participants who receive the RAPT training have fewer crashes in the 6 months following training relative to participants who receive the placebo training, the research team will use Poisson regression to predict the number of crashes using training group, age, sex, and jurisdiction as predictors. Further, given that the previous NHTSA study “Evaluation of the safety benefits of the risk awareness and perception training program for novice teen drivers” (Report No. DOT HS 812 235) found a reduction in crashes following RAPT training only for males, the research team will also include as a predictor an interaction between training group and sex. Other analysis techniques will include survival analysis (e.g., Cox regression) for time data, such as the time until the first crash or citation, or chi-square tests to determine differences in the types of crashes experienced by the RAPT or placebo training groups (e.g., by severity or at-fault). The research team will also examine differences in performance on pre-training, post-training, and follow-up tests of hazard recognition (e.g., using ANOVA). Finally, the research team will determine whether participants differ on the collected measure of exposure (total number of miles driven in a week) by age, sex, or months post-licensure. If differences exist, exposure will be included in any analysis of the effects of training on crashes, citations, or other driving measures.

Table 1. Main research questions and statistical tests for each question.

Research Question(s)	Population	Dependent Variable	Independent Variables	Analysis Method	Data Source
Does RAPT training reduce real-world risky traffic safety behaviors? Do any training-related effects depend on participants’ sex?	All participants in both jurisdictions (n = 10,000)	Count of crashes (or citations) in 6-month period following training	Between-subjects: training group (RAPT, placebo), age, sex, jurisdiction Interactions: training group X sex	Poisson regression	DMV: crashes, citations Self-report: age, sex Experimenter: training group, jurisdiction
		Time to first crash (or citation) in 6-month period following training	Between-subjects: training group (RAPT, placebo), age, sex, jurisdiction Interactions: training group X sex	Survival analysis (Cox regression)	DMV: crashes, citations Self-report: age, sex Experimenter: training group, jurisdiction
Does RAPT training improve performance on tests of hazard recognition?	All participants in both jurisdictions (n = 10,000)	Performance (e.g., average number of hits) on pre-training and post-training hazard recognition tests	Between-subjects: training group (RAPT, placebo), age, sex, jurisdiction Within-subjects: time of test (pre-training, post-training) Interactions: training group X time of test	Mixed-design ANOVA	Self-report: age, sex Experimenter: performance on hazard recognition test, training group, jurisdiction
Are any improvements on tests of hazard recognition still evident at 6-months post-training?	Participants in primary jurisdiction (n = 7,500)	Performance (e.g., average number of hits) on pre-training, post-training, and 6-month follow-up hazard recognition tests	Between-subjects: training group (RAPT, placebo), age, sex, jurisdiction Within-subjects: time of test (pre-training, post-training, 6-month follow-up) Interactions: training group X time of test	Mixed-design ANOVA	Self-report: age, sex Experimenter: performance on hazard recognition test, training group, jurisdiction
Does exposure over the 6-month follow-up period differ by participants’ age or sex?	Participants who complete one-week driving log (n = 2,000)	Total number of miles driven in a week	Between-subjects: month (1-6), age, sex	Between-subjects ANOVA	Self-report: miles driven in a week, age, sex Experimenter: training group, month

B.5. Provide the name and telephone number of individuals consulted on statistical aspects of the design.

The following individuals have reviewed technical aspects of this research plan:

Kathy Sifrit, PhD

Research Psychologist, NHTSA

202-366-0868

Ward Vanlaar, PhD

Statistician, Traffic Injury Research Foundation (TIRF)

613-238-5235

1 For calculations of power, we assumed the same mean crash rate (.089) for the placebo training group as that reported in the previous NHTSA RAPT study (Report No. DOT HS 812 235), and we assumed the standard deviation to be the average of that calculated from the RAPT and placebo training groups (.032) in the same study.

2 Dillman, D. (2007). Mail and internet surveys: The tailored design method. Hoboken: John Wiley and Sons.

3 Mayhew, D.R., Marcoux, K., Wood K., Simpson, H., Vanlaar, W., Lonero, L., & Clinton, C. (2014). Evaluation of driver education in Manitoba and Oregon. Washington, D.C.: American Automobile Association for Traffic Safety.