L32C Phase 2 Supporting Statement Part B_060815

The Supporting Statement

Part B. Collections of Information Employing Statistical Methods

1. Describe potential respondent universe and any sampling selection method to be used.

The potential respondent universe consists of individuals who received traffic incident management training through the National TIM Responder Training Program. These individuals are trainers, responders, and stakeholders of local and state government agencies providing TIM services, including departments of transportation, police, fire, and EMS agencies. The estimated population for Train-the-Trainer students is 5,000 and for the Responders that they train 100,000. Subgroups are respondent type (transportation, police, fire, towing, EMS, other) and training course type (responder training and Train-the-Trainer sessions).

The sampling method used for Train-the-Trainer and Responder Training groups is cluster sampling.  Classes will be randomly selected and all participants administered level 1 and level 2 instruments.

2. Describe procedures for collecting information, including statistical methodology for stratification and sample selection, estimation procedures, degree of accuracy needed, and less than annual periodic data cycles.

Information will be collected in-person for the live training sessions, and electronically (using a software as a service [SAAS] approved survey provider with FedRAMP ATO certification) for online training. The Level 1 assessments will be collected in-person at the training, with the Level 2 assessment including both pre- and post-training assessments. Using a 95% confidence level and a confidence interval of 4%, the required sample size is estimated at 537 and 597 respectively for Train the Trainer and Responder groups.

The objective of the confidence level is to estimate that the sample represents how often the true percentage of the population who would pick an answer. Levels 1, 2, and 3 employ Random sampling using clusters.  Level 4 uses quota sampling.  The estimate for response rate is simply the best guess that the team has given experiences with responders.  The team will try to minimize unit nonresponse bias by encouraging participation though email solicitation and strategic reminders.  Additionally the use of the tool will hopefully make participation easy and thereby encourage participation.

The SAAS tool will be used for the Level 3 and 4 assessments. Level 3 will be administered 90 days after taking the class, via email solicitation, and a sample size of 600 responses are sought for each population. Level 4 agencies will be solicited via email sent to organizations representative of all 50 US states plus the District of Columbia (DC) and Puerto Rico. Approximately 200 agencies will be solicited for L4 (transportation, police, fire, and towing disciplines for each state), with the objective to obtain at least 1 response from each state plus DC and Puerto Rico. Level 4 assessment will be analyzed descriptively.

3. Describe methods to maximize response rate.

The project will employ quota sampling, soliciting participation from the four main disciplines (police, fire, transportation, and towing) in each state. Participants will complete the Level 1 and 2 assessments as part of their training. Since level 1 and level 2 are captive student audiences, we can simply select 600 random participants to achieve the required sample sizes for Train the Trainer and Responder populations. Outreach will be performed to encourage completion of the Level 3 and 4 assessments. A one-page fact sheet and slide deck will be provided to FHWA. This will be accompanied by a 90-minute webinar explaining the tool and other aspects of the evaluation project.

The Level 3 assessment will be solicited through contact with those respondents who have completed Level 1 and 2 assessments. It is estimated that a 25% response rate is possible, therefore 2,400 Level 1 and/or Level 2 students must be engaged to achieve the targeted 600 samples.

Level 4 surveys require more in-depth information to evaluate performance, therefore potential respondents will be provided with notice of the information requested before the assessment. Level 4 will engage the 4 primary responder disciplines (transportation, police, fire, tow/other) from each state (200 total). Due to the variability in responses by state and the limited candidate pool for completing level 4 (agency heads or similar) the research will not attempt to apply statistical sampling methods.

No payment or other direct incentives will be used.

4. Describe tests of procedures or methods.

The assessments will be evaluated prior to implementation through pilot testing and limited field testing. Other than these development efforts, there are no other tests of procedures or methods. Between February 23 and April 30, 2015 approximately 300 Train-the-Trainer and Responder students undertook L1 and L2 assessments during in person training sessions as part of normal workshop procedures. . . During May 2015, the L1 instrument was determined sufficient and the L2 instrument underwent minor modifications. The L3 instrument will be tested by approximately 9 participants between May 1 and June 30, 2015 with any modifications made by July 1, 2015. The L4 instrument will be tested during June 2015 by approximately 9 participants with modifications made by July 31, 2015. The pilot studies will validate the L1 instrument and resulted in minor modifications to several items in the L2 instrument.

5. Provide name and telephone number of individuals who were consulted on statistical aspects of the IC and who will actually collect and/or analyze the information.

• Rebecca Brewster, 770-432-0628

• Grady Carrick, 904-705-8046

• Justin Morgan, 206-528-3243

3