081315 L32C Phase 2 Supporting Statement Final (2)

National Traffic Incident Management (TIM) Responder Training Program Supporting Statement

Introduction:

The Federal Highway Administration (FHWA) is requesting clearance for a new information collection (IC) to collect feedback on, validate the effectiveness of, and assess the value of the National Traffic Incident Management (TIM) Responder Training Program.

The TIM Responder Training Program addresses critical safety and operations concerns of incident management. This IC supports the TIM Responder Training Program by enabling FHWA to collect feedback from the individuals and organizations participating in this national training initiative. FHWA will use the information collected to analyze the effectiveness of the program in improving safety and reducing the congestion associated with highway incident response.

Part A. Justification.

1. Circumstances that make collection of information necessary:

The TIM Responder Training Program was initially developed by the Transportation Research Board (TRB) through the Second Strategic Highway Research Program (SHRP2), an applied research program authorized by the Unites States Congress in the Safe, Accountable, Flexible, Efficient, Transportation Equity Act, a Legacy for Users (SAFETEA-LU), Section 5210 (Public Law 109-59), reauthorized in Moving Ahead for Progress in the 21st Century (MAP-21), Sections 52003 and 52005 (Public Law 112-141), and cooperatively executed by FHWA, TRB, and the American Association of State Highway and Transportation Officials (AASHTO). The FHWA Office of Operations assumed lead implementation responsibility for the TIM Responder Training Program following the completion of the TRB-lead research phase, as jointly agreed by FHWA, TRB, and AASHTO.

The training curriculum, developed through SHRP2 project numbers L12 and L32A, is designed to reach as many responders as possible through in-person training. In the summer of 2012, the FHWA Office of Operations assumed lead implementation responsibility for the in-person training program, and is currently conducting Train-the-Trainer (TtT) sessions throughout the U.S. Additionally, in October 2014 the Office of Operations launched the Web-based Training (WBT) version of the course through its training arm, the National Highway Institute (NHI). The WBT, which was developed under SHRP2 project L32B, is intended to significantly expand the reach of the program, reaching thousands of additional responders. When fully-deployed, the TIM Responder Training Program will produce a cadre of well-trained responders in each State, able to more quickly reduce the time it takes to clear accidents, offering the benefits of reduced congestion and lost travel time for travelers, as well as improved safety conditions for incident responders and motorists.

The SHRP2 program also identified the need for comprehensive evaluation of the benefits of TIM responder training, and developed an electronic post-course assessment tool (Assessment Tool) through project L32C, to be used to gather and analyze survey information related to TIM responder training. The Assessment Tool is described in the SHRP2 Reliability Project L32C pre-draft report (available electronically at http://onlinepubs.trb.org/onlinepubs/shrp2/SHRP2_L32CprepubReport.pdf).

The FHWA Office of Operations requests clearance to survey the individuals and organizations participating in the TIM Responder Training Program in order to collect the information necessary to perform an evaluation of the program in a manner consistent with the framework and tool described in the SHRP2 Reliability Project L32C draft report.

The TIM Responder Training Program and the related program evaluation effort support the United States Department of Transportation (DOT) goals of Safety, Mobility, and Economic Growth. Efficient management of traffic incidents results in increased safety of responders and motorists with a reduction in secondary crashes. Reductions in incident clearance times contribute to reliable mobility and enables economic growth through more efficient movement of goods and services.

2. How, by whom, and for what purpose is the information used:

Information will be collected by the FHWA Office of Operations and its contractors via the Assessment Tool using electronic surveys and, in limited instances, via paper forms, for the purpose of assessing the impacts and effectiveness of the TIM Responder Training Program.

The Assessment Tool will employ a four-level “Kirkpatrick Model” evaluation methodology with survey data collection during and following both in-person and WBT events.¹ Consistent with the Kirkpatrick Model, the Office of Operations intends to survey training participants, their supervisors, and agency administrators/senior management. A demographics survey will be included in surveys for each level.

• Demographics Survey enables the program to gather basic information about the individuals (e.g., business contact information and discipline) who are participating in the evaluation surveys in Levels 1-4. These questions will be administered to participants in all levels of evaluation.

• Level 1 is a reaction survey administered immediately following the training session. This level allows the program to gather feedback from users on the training course they just participated in. These surveys will be administered either in electronic or hardcopy format. There are two versions of this survey, one for responder training sessions and one for TtT sessions.

• Level 2 is administered as part of the training, immediately before and after the training session. These surveys are designed to gauge short- term student learning and retention of course concepts. The surveys will be administered either in electronic or hardcopy format. There are two versions of this survey, one for responder training sessions and one for TtT sessions.

• Level 3 is a behavior assessment, conducted at least three months following the completion of the training sessions. This phase is designed to assess changes in responder behavior, the relevance of those changes to improved incident response, and their sustainability over time. Information will be collected via survey of training participants and their supervisors. Supervisor feedback is essential to obtaining objective and reliable assessments of trainee behavior change. All information will be collected via electronic surveys. The phrasing of questions will vary slightly between the participant and the supervisor versions of the survey.

• Level 4 assesses organizational change resulting from the training program in the medium and long-terms. Surveys will be distributed electronically to senior management officials of trainee organizations. Surveys will be conducted at least three months after training sessions to gauge long-term effects of the training program. All information will be collected via electronic surveys.

Table 1: Time Requirements by Survey Instrument – Incident Responder Training

Survey Instrument	Estimated Time Requirement
Demographics	2 minutes
Level 1	8 minutes
Level 2 – Pre-Test	7.5 Minutes
Level 2 – Post-Test	15 minutes
Level 3	13 minutes
Level 4	*

*Level 4 surveys represent outcomes and are captured in Train the Trainer Sample

Table 2: Time Requirements by Survey Instrument – TtT Training

Survey Instrument	Estimated Time Requirement
Demographics	2 minutes
Level 1	10 minutes
Level 2 – Pre-Test	15 minutes
Level 2 – Post-Test	30 minutes
Level 3 (Supervisor)	13 minutes
Level 4	22 minutes

3. Extent of automated information collection:

The FHWA Office of Operations will provide the option of submitting information electronically for all instances of this IC, and electronic submission will be the preferred method. In some cases, paper forms may be offered (in addition to the electronic option) for the collection of Level 1 and Level 2 information during and immediately following the in-person training sessions. All other surveys will be executed exclusively through an online system. Response to all surveys will be optional.

Participants will be surveyed during or shortly after they attend a TIM training session. Follow-up surveys (Levels 3 and 4) will be distributed to trainees and their supervisors to gauge long-term retention of training materials and the impacts of the training on incident management performance.

Respondents: The Office of Operations plans to collect 12,200 surveys in order to establish a representative sample size spanning all four levels of evaluation and for all formats of the training. Surveys will be collected from two distinct sample groups; individuals that participate in responder training sessions and individuals that participate in TtT sessions. 9,600 surveys will be collected from participants in the responder training sessions, and an additional 2,600 surveys will be collected from participants in the TtT sessions (Tables 3 and 4).

Some individuals may be surveyed more than once in order to cover successive evaluation levels. Additionally, participants may be asked to complete surveys more than once to measure long-term impacts and outcomes (i.e., annually complete Level 2, 3, or 4 surveys).

Table 3: Responder Training Sessions Sample Size by Kirkpatrick Model Level

Kirkpatrick Model Level	Sample Size
1	2,400
2 (Pre-Test)	2,400
2 (Post-Test)	2,400
3	2,400
4	*
Total	9,600

*Level 4 surveys represent outcomes and are captured in Train the Trainer Sample

Table 4: Train the Trainer Sessions Sample Size by Kirkpatrick Model Level

Kirkpatrick Model Level	Sample Size
1	600
2 (Pre-Test)	600
2 (Post-Test)	600
3	600
4	200
Total	2,600

4. Efforts to identify duplication:

The information needed to evaluate the effectiveness of the TIM Responder Training Program can only be collected from the individuals and organizations who participate in the training program. No source of information that can be used for this purpose currently exists.

5. Efforts to minimize the burden on small businesses:

This collection does not affect small businesses.

6. Impact of less frequent collection of information:

The TIM Responder Training Program evaluation is designed to only collect information from participating individuals and organizations necessary to conduct a four-phase Kirkpatrick Model evaluation. Statistical techniques will be employed to minimize the number of individuals who must be surveyed in order to gather the data needed to assess the impact of the training program in a robust manner. Less frequent collections of information would result in an incomplete assessment of the impacts of the TIM Responder Training Program and reduce the ability of the Office of Operations to modify and adapt the program to meet the needs of the traffic incident responder community and to demonstrate the effectiveness of the program.

7. Special circumstances:

FHWA is not requesting any special circumstances.

8. Compliance with 5 CFR 1320.8:

A Federal Register Notice published on August 1, 2014 (FR 79/148 page 44960) solicited public comments. No comments were received.

9. Payments or gifts to respondents:

No payments or gifts will be provided to respondents.

10. Assurance of confidentiality:

Respondents will not be provided with any assurance of confidentiality.

11. Justification for collection of sensitive information:

This IC will not collect sensitive or private information.

12. Estimate of burden hours for information requested:

The Office of Operations intends to collect a total of 12,200 surveys from approximately 6,100 individuals (some individuals will participate in multiple levels of evaluation).

Some individuals will participate in multiple levels of evaluation, which will occur on the same day, or at least 90 days following attendance at a training session (depending on which level of evaluation). Demographic survey questions will be included in each collection interaction, and the time for completing the demographic survey is included in the time and burden estimates for each level below in Table 5. The total burden hours for this information collection are estimated to be 2,900 hours.

Table 5: Breakdown of Estimated Burden Hours

	Survey	# of Respondents	Time (Min)	Burden Hours
Level 1	Responder	2400	8	320
	Train-the-Trainer	600	10	100
Level 2	Responder
	Pre-Course	2400	7.5	300
	Post-Course	2400	15	600
	Train-the-Trainer
	Pre-Course	600	15	150
	Post-Course	600	30	300
Level 3	Responder	2400	13	520
	Train-the-Trainer	600	13	130
Level 4	Train-the-Trainer	200	22	73.3
Demographic	All Surveys	12200	2	406.7
Total				2900

The majority of the participants in the TIM Responder Training Program are highway incident first responders (e.g., police, fire, EMS, and highway civil engineers). Therefore, the FHWA Office of Operations estimates that the total burden hours represent an equivalent cost of $76,589. This is based on the average of the Bureau of Labor Statistics May 2013 estimate of mean hourly wage for the occupational groups which represent highway first responders ($26.41/hour).

13. Estimate of total annual costs to respondents:

Because all surveys will be conducted online, or with paper forms that will be collected during in-person events, there will be no costs for respondents to participate in the IC, outside of the hourly wages outlined in Q12.

14. Estimate of cost to the Federal government:

The cost to the Federal government for this information collection is estimated to be $240,400. This estimate includes 5 hours per month of Federal labor and associated overhead, and payments to contractors who will execute the evaluation, produce reports for FHWA via the automated system. Table 6 below provides a breakdown of the estimate.

Table 6: Government Cost of the Information Collection

Item	Cost
Federal Employee Labor (GS-13/14 level)	$27,000
Contractor: Execution of the Evaluation	$85,500
Contractor: Reports and Reporting	$91,900
Total Cost	$204,400

15. Explanation of program changes or adjustments:

This is a new IC request supporting a new program.

16. Publication of results of data collection:

The FHWA Office of Operations plans to publish information from the evaluation of the TIM Responder Training Program 1-2 times per year for the duration of the program. This information will be published electronically and made available on program websites. Reports will include summary information only and will be designed to demonstrate the effectiveness of the program in achieving program goals.

17. Approval for not displaying the expiration date of OMB approval:

FHWA is not seeking approval to not display the OMB expiration date.

18. Exceptions to certification statement:

FHWA is not seeking an exception to the certification statement.

1 Kirkpatrick, D.L., & Kirkpatrick, J.D. (2007). Implementing the Four Levels. Oakland, CA: Berrett-Koehler Publishers

9