SSA_ConfCloseCall0574

SUPPORTING JUSTIFICATION

Confidential Close Call Reporting System Evaluation-Related Interview Data

Collection

OMB No. 2130-0574

Summary

• This is a revision to a previously approved collection of information.

• The total number of burden hours requested for this submission is 110 hours per year.

• Total number of responses for this submission is 220 per year.

• Total adjustments for this submission decreased by 132 hours, due to a reduction in the number of respondents that may be interviewed.

• There are no program changes.

** The answer to question number 12 itemizes the hourly burden associated with each requirement of this rule (See p. 5).

1. Circumstances that make collection of the information necessary.

In the U.S. railroad industry, injury rates have been declining over the last 25 years. Indeed, the industry incident rate fell from a high of 12.1 incidents per 100 workers per year in 1978 to 3.66 in 1996. As the number of incidents has decreased, the mix of causes has also changed toward a higher proportion of incidents that can be attributed to human and organizational factors. This combination of trends – decrease in overall rates but increasing proportion of human factors-related incidents – has left safety managers with a need to shift tactics in reducing injuries to even lower rates than they are now.

In recognition of the need for new approaches to improving safety, FRA has instituted the Confidential Close Call Reporting System (C³RS). The operating assumption behind C³RS is that by assuring confidentiality, employees will report events which, if dealt with, will decrease the likelihood of accidents. C³RS, therefore, has both a confidential reporting component and a problem analysis/solution component. C³RS is expected to affect safety in two ways. First, it will lead to problem solving concerning specific safety conditions. Second, it will engender an organizational culture and climate that supports greater awareness of safety and a greater cooperative willingness to improve safety.

If C³RS works as intended, it could have an important impact on improving safety and safety culture in the railroad industry. While C³RS has been developed and implemented with the participation of FRA, railroad labor, and railroad management, there are legitimate questions about whether it is being implemented in the most efficacious way, and whether it will have its intended impact. Further, even if C³RS is successful, it will be necessary to know if it is successful enough to implement on a wide scale. To address these important questions, FRA is implementing a formative evaluation to guide program development, a summative evaluation to assess impact, and a sustainability evaluation to determine how C³RS can continue after the test period is over. The evaluation is needed to provide FRA with guidance as to how it can improve the program, and how it might be scaled up throughout the railroad industry.

Program evaluation is an inherently data driven activity. Its basic tenet is that, as change is implemented, data can be collected to track the course and consequences of the change. Due to the setting in which C³RS is being implemented, that data must come from the railroad employees (labor and management) who may be affected. Critical data include beliefs about safety and issues related to safety, and opinions/observations about the operation of C³RS.

This renewal submission addresses the continuation of conducting interviews that constitute one part of an evaluation that will include interview data, surveys of organizational safety culture, review of corrective action documents, and analysis of corporate data on safety. The interviews are needed to provide insight as to how the survey and safety data should be interpreted and insight as to how the workers perceive the program. This renewal submission is requesting a renewal of OMB approval to continue to do interviews.

2. How, by whom, and for what purpose the information is to be used.

The information collected is part of a ten-year (2007-2016) demonstration project to improve rail safety, and thus this study is ongoing. The information collected from this study is being used to identify safety issues and propose corrective action based on voluntary reports of close calls submitted to the Bureau of Transportation Statistics (BTS) and National Aeronautics and Space Administration (NASA). The information collected is aimed at engendering an organizational culture and climate throughout the rail industry that supports greater awareness of safety and a greater cooperative willingness to improve safety to reduce injury rates from incidents caused by human and organizational factors.

Additional interview will be conducted with the two main or key groups – (1) key stakeholders to the process such as FRA officials, industry labor, and carrier management within participating railroads; and (2) employees in participating railroads who are eligible to submit closes calls to the C³RS. The data will be used by FRA’s contractors to further evaluate the process by which the C³RS was implemented. Specifically, data collected by the evaluation’s prime contractor (Jacobs.), sub-contractor (Fulcrum Corporation) and by staff at the Volpe National Transportation Systems Center will be used to determine how the program is being implemented and participants’ views on lessons learned.

To reduce the chances of getting skewed data, the evaluators ask for participants with positive and negative views of C³RS and have designed the questions to collect both positive and negative information about C³RS. For example, the key stakeholders’ first question asks about both positive and negative events that have occurred. Another question asks if any groups could change to improve C³RS. The railroad employee interviews also ask for suggestions for improvements.

2. Extent of automated information collection.

FRA strongly endorses and highly encourages the use of advanced information technology, where feasible, to reduce burden. This study is ongoing and, as noted previously, computers are not the most appropriate tool for the needed data collection. Opinions about safety, C³RS, and related topics can be most comprehensively and clearly collected by means of phone interviews by trained staff. The questions are inherently long as people are being asked for opinions that cannot be reduced to fixed-choice formats. In order to avoid the burden of asking people to write long answers, interviews, in which the trained interviewers record responses, are necessary. Further, any given respondent may have a unique perspective or opinion that deserves clarification or follow-up elaboration to a specific question(s). This process cannot be predicted well enough in advance to allow the standardization that is conducive to computerized administration by FRA’s contractors.

Since the requested burden is already minimal (i.e., 110 hours per year), it is unlikely that use of advanced information technology would further reduce this burden to any significant degree.

4. Efforts to identify duplication.

The information collection requirements to our knowledge are not duplicated anywhere.

Similar data are not available from any other source.

5. Efforts to minimize the burden on small businesses.

Small businesses and entities are not involved. Additional interviews of individuals will be conducted to evaluate the implementation of the C³RS. Thus, there is no impact at all on small entities that will result from this collection of information.

6. Impact of less frequent collection of information.

If this continuing collection of information is not conducted, FRA will lose a unique and valuable opportunity to further study and evaluate the C³RS and the issue of railroad worker injuries. FRA’s main mission is to promote and enhance rail safety throughout this country. Over the last several years, FRA has made a major policy commitment to improving safety in the railroad industry through innovative, human factors based programs. The agency has backed up this commitment with major funding for the C³RS program. The operating assumption behind C³RS is that by assuring confidentiality, employees will report events which, if dealt with, will decrease the likelihood of accidents. The evaluation of this program is designed to provide FRA with the guidance it needs regarding the expansion of this pilot program throughout the rail industry in order to reduce the number and severity of rail accidents/incidents that occur each year and the corresponding injuries and fatalities that rail workers experience. Without this needed guidance, the quality of the agency’s C³RS decision will be severely degraded. Further, without this collection of information, FRA will be greatly hampered in meeting Departmental and agency goals to reduce rail worker injuries and fatalities caused by human factor errors.

The frequency of data collection is scheduled to conform to a methodology that will allow FRA’s contractors to track change over time. Fewer data collection points than the ones proposed will make it difficult to discern patterns.

In sum, this collection of information serves both DOT’s and FRA’s top goals, which is to improve transportation/rail safety in the United States.

7. Special circumstances.

All information collection requirements are in compliance with this section.

8. Compliance with 5 CFR 1320.8.

As required by the Paperwork Reduction Act of 1995, FRA published a 60 day notice in the Federal Register on November 25, 2013, soliciting comment on this particular information collection. 78 FR No. 227. FRA received no comments in response to this notice.

9. Payments or gifts to respondents.

There are no monetary payments or gifts made to respondents associated with the information collection requirements contained in this regulation.

10. Assurance of confidentiality.

As noted in the previous submission, FRA fully complies with all laws pertaining to confidentiality, including the Privacy Act of 1974. Thus, information obtained or acquired by FRA’s contractor, Jacobs, is used exclusively for analysis of the implementation, impact, and sustainability of the C³RS program. None of the information obtained that might identify individuals is disseminated or disclosed in any way. This fact has been explained to earlier respondents and will be explained to additional interviewees prior to beginning any interview. No micro-level data is or will be released to the public. All reported data is and will be aggregated in a manner that prevents identification of a specific individual. Additionally, the interview protocol does not ask participants for their names.

11. Justification for any questions of a sensitive nature.

There are no questions or information of a sensitive nature involved or data that would normally be considered private matters in this information collection.

12. Estimate of burden hours for information collected.

The change in burden is due to an adjustment in the number of respondents that may be interviewed.

Form	Annual Number of Respondents	Number of Responses per Respondent	Time per Response	Total Burden per year
FRA F 6180.126A Key Stakeholder Interviews for the Evaluation of C3RS	160	1	30 min	80hrs
FRA F 6180.126B Railroad Employees Views of C3RS	60	1	30 min	30 hrs

* See explanation below.

Data collection plans estimate a maximum of 660 interviews over a three year period. Each interview will last approximately 30 minutes. Thus, FRA is requesting approval to conduct 330 hours of interviews over the three years, which amounts to an average of 110 hours per year once the proposed collection begins.

Two interview protocols will be used, one for key stakeholders, and one for railroad workers. (Precise question wording will differ slightly depending on the key stakeholder’s position. Any follow up and/or clarification questions will depend on comments made by respondent during the interview. The intent of the protocol and the questions will remain the same.) Due to the frequency of the key stakeholder interviews will depend on stability of local C³RS implementations, it is difficult to provide a specific number of interviews for each group (in the early stages of C³RS, or if the program is not working smoothly, interviews will be more frequent). It is also the case that the number of railroad workers who will be available for interviews will vary with circumstances (e.g., scheduling issues affecting the ability to interview people in groups). However, given what we know from experience about interviewing stakeholders and railroad workers, it is reasonable to assume that less than a quarter of the total number of respondents will be railroad workers, and the rest key stakeholders.

13. Estimate of total annual costs to respondents.

There are no additional costs to respondents who will be interviewed for this collection of information.

14. Estimate of Cost to Federal Government.

Costs for the evaluation contractor are based on the average of BLS rates for Industrial-Organization Psychologist and Industrial Engineer ($43.49). Two people perform each interview. Costs for government personnel are based on published data for GS 14 (in 2012 for Boston $57.4). This includes time for: development of interview protocols (which is minimal because the project is on-going); administration of the interviews including, travel time for the railroad worker interviews; interview time; note taking; and time to analyze the data. A burden rate of 40% was assumed for government personnel and 75% for contract personnel.

Task	Cost for all years
Key stakeholder interviews (development, administration, analysis)	$119,287
Railroad worker interviews (development, administration, analysis)	$79,500
TOTAL COST	$198,787

15. Explanation of program changes and adjustments.

The burden for this collection of information has decreased by 132 hours from the previously approved submission. The change in burden is due to an adjustment in the number of respondents that may be interviewed. Additional railroads are joining the C³RS program are likely to be added to the evaluation. There is no increase in the number of hours as the duration of each interview decreased from 60 to 30 minutes based on past experience. In the previous submission, FRA estimated that a total of 726 respondents would be interviewed over a three-year period or a total of 242 respondents per year. In this submission, FRA has revised the total number of respondents who will be interviewed to 660 or a total of 220 respondents per year. The burden time of thirty minutes per interview has decreased based on past experience on the project (in our experience most interviews last 30 minutes or less).

The current OMB inventory shows a total of 242 hours for this collection of information, while the present submission exhibits a total of 110 hours. Hence, there is a decrease in burden of 132 hours.

16. Publication of results of data collection.

Begin data collection	On going
Data analysis	Ongoing, as data come in
Interim reports – participating railroads	Baseline, midterm, and follow up reports to each railroad and to FRA as data is available Q4 2013 to Q3 2016
Final reports and publications	Q3 2016

Interview data will be content analyzed.

17. Approval for not displaying the expiration date for OMB approval.

FRA is not seeking an exemption. Once OMB approval is received, FRA will publish the approval number for these information collection requirements in the Federal Register.

18. Exception to certification statement.

No exceptions are taken at this time.

Meeting Department of Transportation (DOT) Strategic Goals

This information collection supports the top DOT strategic goal, namely transportation safety. If this collection of information is not conducted, FRA will lose a unique and valuable opportunity to study the Confidential Close Call Reporting System (C³RS) and the issue of railroad worker injuries. FRA’s main mission is to promote and enhance rail safety. Over the last several years, FRA has made a major policy commitment to improving safety in the railroad industry through innovative, human factors based programs. The agency has backed up this commitment with major funding for the C³RS program. The operating assumption behind C³RS is that by assuring confidentiality, employees will report events which, if dealt with, will decrease the likelihood of accidents. The evaluation of this program is designed to provide FRA with the guidance it needs regarding the expansion of this pilot program throughout the rail industry in order to reduce the number and severity of rail accidents/incidents that occur each year and the corresponding injuries and fatalities that rail workers experience. Without this needed guidance, the quality of the agency’s C³RS decision will be severely degraded. Further, without this collection of information, FRA will be greatly hampered in meeting Departmental and agency goals to reduce rail worker injuries and fatalities caused by human factor errors.

The frequency of data collection is scheduled to conform to a methodology that will allow FRA’s contractors to track change over time. Fewer data collection points than the ones proposed will make it difficult to discern patterns.

In sum, this collection of information supports FRA’s mission, which is to promote and enhance rail safety throughout the United States. As always, FRA seeks to do its utmost to fulfill DOT Strategic Goals and to be an integral part of One DOT.

8