ConfCloseCall0574AU

SUPPORTING JUSTIFICATION

Confidential Close Call Reporting System Evaluation-Related Interview Data

Collection

OMB No. 2130-0574

Summary

• This is a revision to a previously approved collection of information.

• The total number of burden hours requested for this submission is 242 hours.

• Total number of responses for this submission is 242.

• Total adjustments for this submission amount to 25 hours.

• There are no program changes regarding the actual information to be collected. The only difference from the previously approved submission pertains to an adjustment in the number of respondents.

** The answer to question number 12 itemizes the hourly burden associated with each requirement of this rule (See p. 5).

1. Circumstances that make collection of the information necessary.

In the U.S. railroad industry, injury rates have been declining over the last 25 years. Indeed, the industry incident rate fell from a high of 12.1 incidents per 100 workers per year in 1978 to 3.66 in 1996. As the number of incidents has decreased, the mix of causes has also changed toward a higher proportion of incidents that can be attributed to human and organizational factors. This combination of trends – decrease in overall rates but increasing proportion of human factors-related incidents – has left safety managers with a need to shift tactics in reducing injuries to even lower rates than they are now.

In recognition of the need for new approaches to improving safety, FRA has instituted the Confidential Close Call Reporting System (C³RS). The operating assumption behind C³RS is that by assuring confidentiality, employees will report events which, if dealt with, will decrease the likelihood of accidents. C³RS, therefore, has both a confidential reporting component and a problem analysis/solution component. C³RS is expected to affect safety in two ways: First, it will lead to problem solving concerning specific safety conditions. Second, it will engender an organizational culture and climate that supports greater awareness of safety and a greater cooperative willingness to improve safety.

If C³RS works as intended, it could have an important impact on improving safety and safety culture in the railroad industry. While C³RS has been developed and implemented with the participation of FRA, railroad labor, and railroad management, there are legitimate questions about whether it is being implemented in the most efficacious way, and whether it will have its intended impact. Further, even if C³RS is successful, it will be necessary to know if it is successful enough to implement on a wide scale. To address these important questions, FRA is implementing a formative evaluation to guide program development, a summative evaluation to assess impact, and a sustainability evaluation to determine how C³RS can continue after the test period is over. The evaluation is needed to provide FRA with guidance as to how it can improve the program, and how it might be scaled up throughout the railroad industry.

Program evaluation is an inherently data driven activity. Its basic tenet is that, as change is implemented, data can be collected to track the course and consequences of the change. Because of the setting in which C³RS is being implemented, that data must come from the railroad employees (labor and management) who may be affected. Critical data include beliefs about safety and issues related to safety, and opinions/observations about the operation of C³RS.

This renewal submission addresses the continuation of conducting interviews that constitute one part of an evaluation that will combine interview data, surveys of organizational safety culture, and analysis of corporate data on safety. The interviews are needed to provide insight as to how the survey and safety data should be interpreted and insight as to how the workers perceive the program. This renewal submission is requesting a renewal of OMB approval to continue to do interviews.

2. How, by whom, and for what purpose the information is to be used.

The information collected is part of a nine-year (2007-2015) demonstration project to improve rail safety, and thus this study is ongoing. The information collected from this study is being used to identify safety issues and propose corrective action based on voluntary reports of close calls submitted to the Bureau of Transportation Statistics (BTS) and National Aeronautics and Space Administration (NASA). The information collected is aimed at engendering an organizational culture and climate throughout the rail industry that supports greater awareness of safety and a greater cooperative willingness to improve safety to reduce injury rates from incidents caused by human and organizational factors.

Additional interview will be conducted with the two main or key groups – (1) key stakeholders to the process such as FRA officials, industry labor, and carrier management within participating railroads and (20 employees in participating railroads who are eligible to submit closes calls to the Confidential Close Call Reporting System. The data will be used by FRA’s contractors to further evaluate the process by which the C³RS was implemented, its impact on the rail industry, and factors related to the sustainability of the program. Specifically, data collected by the evaluation’s prime contractor (Vector Research Center Division of TechTeam Government Solutions) and by staff at the Volpe National Transportation Systems Center will be used to determine whether the program is succeeding, how it can be improved and, if successful, what is needed to spread the program throughout the entire railroad industry.

2. Extent of automated information collection.

FRA strongly endorses and highly encourages the use of advanced information technology, where feasible, to reduce burden. This study is ongoing and, as noted previously, computers are not the most appropriate tool for the needed data collection. Opinions about safety, C³RS, and related topics can be most comprehensively and clearly collected by means of phone interviews by trained staff.

There are two types of interviews: Implementation (every other month by phone) and Phased (baseline, midterm, follow-up on site). The phase interviews occur in person. The purpose of being on site is to get information from a broad grouping of people who are typical of the workforce.  Some of these interviews are conducted with people not working in the C3RS project and are scheduled informally on the day of the interview.   The only way to do this is to catch them at convenient times in their own work schedules. The railroad does not know in advance who specifically will be working on what day and when they will come to work as most people work on “on call” shifts coming to work when the train is ready to be driven to the next location.  This is common to the railroad industry.

The questions are inherently long as people are being asked for opinions that cannot be reduced to fixed-choice formats. In order to avoid the burden of asking people to write long answers, interviews, in which the trained interviewers record responses, are necessary. Further, any given respondent may have a unique perspective or opinion that deserves clarification or follow-up elaboration to a specific question(s). This process cannot be predicted well enough in advance to allow the standardization that is conducive to computerized administration by FRA’s contractors.

Since the requested burden is already minimal (i.e., 242 hours), it is unlikely that use of advanced information technology would further reduce this burden to any significant degree.

4. Efforts to identify duplication.

The information collection requirements to our knowledge are not duplicated anywhere.

Similar data are not available from any other source.

5. Efforts to minimize the burden on small businesses.

Small businesses and entities are not involved. Additional interviews of individuals will be conducted to evaluate the effectiveness of the Confidential Close Call Reporting System. Thus, there is no impact at all on small entities that will result from this collection of information.

6. Impact of less frequent collection of information.

If this continuing collection of information is not conducted, FRA will lose a unique and valuable opportunity to further study and evaluate the Confidential Close Call Reporting System (C³RS) and the issue of railroad worker injuries. FRA’s main mission is to promote and enhance rail safety throughout this country. Over the last several years, FRA has made a major policy commitment to improving safety in the railroad industry through innovative, human factors based programs. The agency has backed up this commitment with major funding for the C³RS program. The operating assumption behind C³RS is that by assuring confidentiality, employees will report events which, if dealt with, will decrease the likelihood of accidents. The evaluation of this program is designed to provide FRA with the guidance it needs regarding the expansion of this pilot program throughout the rail industry in order to reduce the number and severity of rail accidents/incidents that occur each year and the corresponding injuries and fatalities that rail workers experience. Without this needed guidance, the quality of the agency’s C³RS decision will be severely degraded. Further, without this collection of information, FRA will be greatly hampered in meeting Departmental and agency goals to reduce rail worker injuries and fatalities caused by human factor errors.

The frequency of data collection is scheduled to conform to a methodology that will allow FRA’s contractors to track change over time. Fewer data collection points than the ones proposed will make it difficult to discern patterns.

In sum, this collection of information serves both DOT’s and FRA’s top goals, which is to improve transportation/rail safety in the United States.

7. Special circumstances.

All information collection requirements are in compliance with this section, except for the interviews every other month by phone. Thus, this collection of information does not necessitate respondents to prepare a written response to this collection of information in fewer than 30 days (no written responses are involved, only voluntarily participation in phone interviews). This collection of information does not have respondents submitting more than one copy of an original document and two copies of that document (again they are submitting no documents because the interviews are conducted by phone).

This collection of information is not conducted in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of the study. This collection of information does not require the use of a statistical data classification not been reviewed and approved by OMB and does not include a pledge of confidentiality not supported by authority established in statute. Finally, this collection of information does not require respondents to submit proprietary trade secrets or other confidential information.

8. Compliance with 5 CFR 1320.8.

As required by the Paperwork Reduction Act of 1995, FRA published a notice in the Federal Register on July 13, 2010, soliciting comment on this particular information collection. 75 FR 40021. FRA received no comments in response to this notice.

The required second Notice (30-day) was published in the Federal Register on September 21, 2010. See 75 FR 57546. Commenters were asked to submit any comments concerning the renewal of this collection of information and associated burde costs and burden estimates directly to OMB using the format – paper or electronic – most convenient to them.

9. Payments or gifts to respondents.

There are no monetary payments or gifts made to respondents associated with the information collection requirements contained in this regulation.

10. Assurance of confidentiality.

As noted in the previous submission, FRA fully complies with all applicable laws and regulations pertaining to confidentiality. FRA and its contractors protect confidentiality by the following: not recording names or other identifying information; marking documents with FOIA language “For Official Use Only”; locking up paper notes until transcribed then shredding them; encrypting electronic files; and only reporting summarized information. FRA and its contractors also have a non-disclosure agreement with railroads if there is a request for any of their company confidential data.

In addition, FRA’s contractor lawyers have cited 49 CFR Part 9.5, which provides that:,  “No employee of the Department may provide testimony or produce any material contained in the files of the Department, or disclose any information or produce any material acquired as part of the performance of that employee’s official duties or because of that employee’s official status unless authorized in accordance with this Part, or by other applicable law.”

Thus, information obtained or acquired by FRA’s contractors, TechTeam Government Solutions and Volpe, is used exclusively for analysis of the implementation, impact, and sustainability of the C³RS program. None of the information obtained that might identify individuals is disseminated or disclosed in any way. This fact has been explained to earlier respondents and will be explained to additional interviewees prior to beginning any interview. No micro-level data is or will be released to the public. All reported data is and will be aggregated in a manner that prevents identification of a specific individual. Additionally, the interview protocol does not ask participants for their names, or the name of the company they work for. (Note: The Privacy Act does not apply to this particular information collection because the Privacy Act concerns the protection of individuals.  In this study, FRA and its contractors collect no personal, subject level data that can be connected to a given individual as we neither ask for nor retain names or employee numbers.)

11. Justification for any questions of a sensitive nature.

There are no questions or information of a sensitive nature involved or data that would normally be considered private matters in this information collection.

12.        Estimate of burden hours for information collected.

Form	Annual Number of Respondents	Number of Responses per Respondent	Time per Response	Total Burden per year
FRA F 6180.126A Key Stakeholder Interviews for the Evaluation of C3RS	192	1	60 min	192
FRA F 6180.126B Railroad Employees Views of C3RS	50	1	60 min	50

* See explanation below.

Data collection plans estimate a maximum of 726 interviews. Each interview will last approximately one hour. Thus, FRA is requesting approval to conduct 726 hours of interviews over the three years, which amounts to an average of 242 hours per year once the proposed collection begins.

Two interview protocols will be used, one for key stakeholders, and one for railroad workers. (Precise question wording will differ slightly depending on the key stakeholder’s position, but the intent of the protocol and the questions will remain the same.) Because the frequency of the key stakeholder interviews will depend on stability of local C³RS implementations, it is difficult to provide a specific number of interviews for each group. (In the early stages of C³RS, or if the program is not working smoothly, interviews will be more frequent.) It is also the case that the number of railroad workers who will be available for interviews will vary with circumstances (e.g., scheduling issues affecting the ability to interview people in groups). However, given what we know from experience about interviewing stakeholders and railroad workers, it is reasonable to assume that less than a quarter of the total number of respondents will be railroad workers, and the rest key stakeholders.

13. Estimate of total annual costs to respondents.

There are no additional costs to respondents who will be interviewed for this collection of information.

14. Estimate of Cost to Federal Government.

Costs for the evaluation contractor are based on BLS rates for management analysts ($40.70). Two people perform each interview. Costs for government personnel are based on published data for GS 13 – GS 15 (in 2010 for Boston $41.06-$75.51 for DC $40.86-$74.51). This includes time for: development of interview protocols (which is minimal because the project is on-going); administration of the interviews including, travel for the railword worker interviews; interview time; note taking; and time to analyze the data. A burden rate of 40% was assumed.

Task	Cost for all years
Key stakeholder interviews (development, administration, analysis)	$61,429
Railroad worker interviews (development, administration, analysis)	$110,950
TOTAL COST	$172,379

15. Explanation of program changes and adjustments.

The burden for this collection of information has decreased by 25 hours from the previously approved submission. The change in burden is due to an adjustment in the number of respondents interviewed. In the previous submission, FRA estimated that a total of 800 respondents would be interviewed over a three-year period or a total of 267 respondents per year. In this submission, FRA has revised the total number of respondents who will be interviewed to 726 or a total of 242 respondents per year. The burden time of one (1) hour per interview remains unchanged.

The current OMB inventory shows a total of 267 hours for this collection of information, while the present submission exhibits a total of 242 hours. Hence, there is a decrease in burden of 25 hours.

Again, it bears mentioning that there is no change to the actual information being collection. The only difference from the previously approved submission pertains to an adjustment in the number of respondents.

There is no change in cost to respondents.

16. Publication of results of data collection.

Begin data collection	ASAP
Data analysis	Ongoing, as data come in
Interim reports – participating railroads	Baseline, midterm, and follow up reports to each railroad and to FRA as data is available Q4 2010 to Q4 2013
Final reports and publications	Q3 2015

Interview data will be content analyzed.

17. Approval for not displaying the expiration date for OMB approval.

FRA is not seeking an exemption. Once OMB approval is received, FRA will publish the approval number for these information collection requirements in a Federal Register Notice.

18. Exception to certification statement.

No exceptions are taken at this time.

Meeting Department of Transportation (DOT) Strategic Goals

This information collection supports the top DOT strategic goal, namely transportation safety. If this collection of information is not conducted, FRA will lose a unique and valuable opportunity to study the Confidential Close Call Reporting System (C³RS) and the issue of railroad worker injuries. FRA’s main mission is to promote and enhance rail safety. Over the last several years, FRA has made a major policy commitment to improving safety in the railroad industry through innovative, human factors based programs. The agency has backed up this commitment with major funding for the C³RS program. The operating assumption behind C³RS is that by assuring confidentiality, employees will report events which, if dealt with, will decrease the likelihood of accidents. The evaluation of this program is designed to provide FRA with the guidance it needs regarding the expansion of this pilot program throughout the rail industry in order to reduce the number and severity of rail accidents/incidents that occur each year and the corresponding injuries and fatalities that rail workers experience. Without this needed guidance, the quality of the agency’s C³RS decision will be severely degraded. Further, without this collection of information, FRA will be greatly hampered in meeting Departmental and agency goals to reduce rail worker injuries and fatalities caused by human factor errors.

The frequency of data collection is scheduled to conform to a methodology that will allow FRA’s contractors to track change over time. Fewer data collection points than the ones proposed will make it difficult to discern patterns.

In sum, this collection of information supports FRA’s mission, which is to promote and enhance rail safety throughout the United States. As always, FRA seeks to do its utmost to fulfill DOT Strategic Goals and to be an integral part of One DOT.

10