ICT Survey Part A_10_24_18

SUPPORTING JUSTIFICATION – Part A

Information and Communications Technology (ICT) Needs Assessment

OMB Control Number 2130-NEW

Summary of Submission:

• This information collection request is a new submission. The proposed assessment is intended to create a better understanding of the information and communications technologies (ICT) needs of specific railroad audiences.

• The estimated total number of burden hours requested for this submission is 127 hours.

• The estimated total number of responses is 382.

• The required 60-Day Federal Register Notice was published in the Federal Register on April 18, 2017, See 82 FR 18341. FRA received no comments in response to this notice.

• The required 30-Day Federal Register Notice was published in the Federal Register on July 18, 2017, See 82 FR 32923. FRA received no comments in response to this notice.

1. Justification

1. Circumstances that make the collection of information necessary.

The Federal Railroad Administration Office of Research, Development and Technology (FRA RD&T), is seeking the approval from the Office of Management and Budget (OMB) to conduct a needs assessment (Scriven & Roth, 1990) survey designed to understand (a) the current use, skills, and attitudes related to information and communications technology (ICT) by railroad employees, and (b) outcomes related to a specific FRA RD&T ICT-based program.

For the purpose of this survey, ICT is defined as technology and tools that railroad employees use to share, distribute, and gather information, and to communicate with one another. Needs assessment is a systematic process used to set priorities and make decisions about program or organizational improvement and resource allocation. Within an area of interest in an organization, needs are measured discrepancies (“gaps”) between the current (“what is”) state of affairs and the desired (“what should be”) state. A prioritized need is a problem or situation that should be remedied or addressed for the good of the organization and the groups it serves (Witkin & Altschuld, 1995; Altschuld & Kumar, 2015).

At present, the ICT preferences of railroad employees is unknown. With this submission, FRA is requesting permission to acquire the needed knowledge through a needs assessment survey.

FRA RD&T uses ICT to disseminate research findings and to increase safety awareness. The data gathered in this survey will help FRA develop an understanding of how to best formulate, distribute, and display information that would be used by railroad employees for educational purposes. Additionally, this survey will gather baseline awareness and outcome data for a particular FRA RD&T ICT-based program.

FRA RD&T supports initiatives that leverage web-based content to reach out to railroad employees. Two such websites are an E-Learning Attention Website and Railroader’s Guide to Healthy Sleep (https://www.fra.dot.gov/Page/P0872, https://www.railroadersleep.org/). The ICT needs assessment will help FRA supplement and modify the content of current educational outreach initiatives, and to plan additional educational initiatives.

This needs assessment survey will be guided by four key questions:

1. How much do Transportation, Yard, and Engine (TY&E) railroaders use ICT?

Understanding how railroad employees use personal and mobile ICT tools (e.g., cell phones and tablets) in their daily lives will inform FRA’s understanding of how to reach this audience with non-regulatory educational and safety promotional material.

For example, online training is increasingly using interactive learning and testing tools. Those who are very comfortable with ICT tools would find online training relatively easy to manage, while other those who are slow to adopt technology may be less comfortable.

The railroad industry is just beginning a major turnover in its demographics; a wave of the most experienced workers are reaching retirement age, and an influx of new hires are entering the industry. While the new railroaders may be new to the job of railroading, they are not necessarily young in age, as many are switching careers. Therefore, this survey will help FRA program planners understand not only the current ICT use, skills and attitudes of railroaders, but also the demographics of this changing population.

Armed with the knowledge of railroader ICT skills, FRA can deliver information through media and tools appropriate to their audience.

2. For what reasons do TY&E railroaders use various ICT?

There are many avenues of information dissemination and sharing. Railroaders may be uncomfortable using or unfamiliar with a variety of ICT. This lack of awareness of specific ICT by railroaders could be critical to the development of FRA programs. For instance, if a lot of railroaders do not know what a blog is, FRA would need to consider whether or not investing significant resources in developing blogs is a wise use of funding.

3. What are TY&E railroader attitudes towards ICT?

Attitude towards ICT is an important predictor for use and acceptance (Spence, DeYoung, & Feng, 2009). Much of the research on ICT use shows that those with positive attitudes are more innovative and have a higher perception of the advantages of ICT (see Verdegem & De Marez, 2011, for a review of ICT research). This is particularly important for designing future communication campaigns and for gaining a clear understanding of baseline TY&E railroader attitudes towards ICT.

4. How do TY&E railroaders use a specific FRA-RD&T ICT-based program?

This question is intended to provide a baseline knowledge of the awareness and use of the FRA RD&T Railroaders’ Guide to Health Sleep website, to gauge the reach of this program against which future communication campaigns can be measured. FRA will be better positioned to design, deliver, and improve such outreach campaigns to reach railroaders once it has an understanding of railroaders’ familiarity with and use of this website.

As organizations rely more and more on ICT, knowledge about the intended users becomes critically important. Program managers who are informed of intended user needs are better able to tailor program resources and efforts to meet those audiences’ needs. For example, the U.S. Postal Service (2013) deploys an annual diary study to understand citizens’ attitudes and use of postal and other forms of communication so that they can best serve the consumer.

2. Indicate how, by whom, and for what purpose the information is to be used.

This is a new collection of information, and is entirely voluntary.

The main objectives in this study are to:

• Determine the ways TY&E Class I railroaders use ICT;

• Describe the skills and attitudes of TY&E Class I railroaders when using ICT; and

• Identify the best approaches for reaching TY&E Class I railroaders in future ICT-based safety education efforts.

• Describe the awareness and use of an existing FRA RD&T ICT-based program.

Primary users will be staff within FRA RD&T, who will have access to summary data. The findings of this study will provide baseline measures for monitoring the industry’s experience with ICT. The findings will also inform FRA RD&T’s research agenda and provide valuable insight for program development. The study should help FRA reach the railroad industry’s diverse population.

The purpose of this research is not to make recommendations or regulations to industry. It is to explore how FRA can best effectively and efficiently communicate information to the railroad population.

3. Extent of automated information collection.

The primary planned distribution method for this study is paper questionnaire mailed through the U.S. Postal Service. We considered all ways to use information technology to reduce the burden of information collection. However, we are planning a mailed questionnaire to avoid the possibility that only online-savvy railroaders would complete an emailed questionnaire (potentially skewing any findings) or, by using a convenience sample, that would reduce the ability to confidently generalize to the industry.

Each letter will also include a URL to the online version of the survey, and each URL will be coded to match the print survey code. Codes allow the participant to be taken off of the follow-up list and coded as “responded”. The online option in the invitation letter eases the burden on the respondent, by not requiring him/her to mail back an envelope, and the research team, because the returned paper survey will not have to be processed and scanned.

Contact information will be provided on each mailing and the U.S. DOT Volpe Center (supporting this FRA RD&T effort) research team will be prepared to respond to participant queries. The web-based version of the survey will be hosted on an Internet survey platform, such as LimeSurvey, Survey Gizmo, Qualtrix, or Survey Monkey. The print version will be created with text recognition software, such as ReMark, to enable scanning the data into electronic format.

3. Efforts to identify duplication.

To FRA’s knowledge, this is the first data collection effort of its kind. Similar studies that cannot be modified for use for FRA’s purposes include:

1. a Federal ICT Survey for expenditures on capitalized and non-capitalized equipment

2. the U.S. Postal Service annual diary survey of households, which includes questions about household Internet access, use, and some attitude items

3. the Federal Census Population Survey, which includes a section on broadband Internet access and use.

Each of these provides some valuable information that may or may not apply to the railroad population. Yet there is no way to isolate (identify and stratify) the Class I railroad TY&E craft worker population specifically. The current proposed study intends to address this gap.

3. Efforts to minimize the burden of small business.

The data collection will not impact small businesses or other small entities.

3. Impact of less frequent collection of information.

FRA RD&T invests in and creates programs for the railroading population to improve safety in the industry. However, these programs are partly dependent upon railroader awareness of them. Without the knowledge gained from this needs assessment survey, the effectiveness of these programs may be compromised due to not being able to reach the target audience as effectively. FRA invests significant resources to build effective programs for improving safety in the railroad industry, therefore any information gap in how to reach the target audience would undermine these efforts.

3. Special circumstances.

This is a one-time information collection; no documents other than the completed survey are requested, and respondents are not asked to retain any records. FRA and its contractor will treat the source of the data as confidential. FRA is not a statistical agency using a contractor to collect these data. Therefore, we cannot offer participants information protection under the Confidential Information Protection and Statistical Efficiency Act (CIPSEA, 44 USC 3501-2). However, FRA fully complies with laws pertaining to confidentiality, specifically the Privacy Act of 1974.

We will collect and handle all information to minimize risk of disclosure, including properly training staff; ensure the data are used only for statistical purposes; review information to be disseminated to prevent identifiable information from being reasonably inferred by either direct or indirect means; and supervise and control agents who have access to the data.

A unique ID number will be assigned to each participant by the contractor. Only the contractor will know the names of the participants and their corresponding ID numbers. The ID number will allow each questionnaire to be tracked so that only non-respondents will receive reminders. Once the questionnaire is closed and the data are coded, the list of participant names and their corresponding ID numbers will be destroyed. Only aggregate results will be reported. De-identified data will also be provided to www.data.gov. No data will be reported by individual or by railroad.

3. Compliance with 5 CFR 1320.8.

As noted in the Summary on page 1 and as required by the Paperwork Reduction Act of 1995, FRA published the required 60-Day Federal Register Notice on April 18, 2017, see 82 FR 18342. FRA received no comments in response to this notice. FRA published the required 30-Day Federal Register Notice on July 18, 2017, see 82 FR 32923.

Experts in survey development and the railroad industry have reviewed and provided input into the current instrument. The survey items have been reviewed by four TY&E labor union leaders. A pilot study will be undertaken only with the intent to improve wording of items and create a more readable instrument for this population, or if any item emerges as redundant or exceptionally problematic, it will be dropped.

For the pilot study, a convenience sample of 20–30 TY&E Class I railroaders, selected by union leaders, will be contacted and asked to complete the survey. If they agree, the introductory packet described herein will be sent to them. Railroaders will be encouraged to provide input on the survey items on a 1-on-1 basis by calling the Volpe Center point of contact. Concurrently, a stakeholder panel of key industry representatives will be convened to review the instrument. This input will be incorporated into the instrument.

As a result of the pilot study, no substantive changes will be made to any items, nor will any additional items be added to the instrument. Therefore, OMB’s approval of the instrument for the ICT study should remain applicable, and no re-review would be warranted.

3. Payment or gift to respondents.

No incentives will be used.

3. Assurance of confidentiality.

The instrument will clearly state in the cover letter and on the first page that participants may choose whether to be in this study or not and if they volunteer to be in this study, they may withdraw at any time without consequences of any kind. Participants will also be informed that any information collected in the study will remain confidential to the extent permitted by law and any reporting will be done in aggregate to maintain confidentiality.

As will be explained in the informed consent, each participant will be assigned a unique identifier when address labels are created. The crosswalk listing of identifier and participant will be encrypted and stored by the FRA contractor in a password-protected, non-networked computer and this list will only be used to track participation in an effort to limit follow up efforts. Upon closing of the survey, any personally identifiable information (PII) will be separated from the data. Analyses will occur without any PII attached. All survey data will be aggregated and no individual data will be reported. All data will be kept on a password-protected computer and access will be limited to the principal investigators and team members who have completed National Institute of Health Human Subjects Research Certification (https://phrp.nihtraining.com/) or another certified training.

Upon completion of the study all PII information linking names with unique identifiers will be deleted or destroyed, including appropriate erasing of electronic data to ensure non-recoverability. FRA fully complies with laws pertaining to confidentiality, specifically the Privacy Act of 1974.

3. Justification for any questions of a sensitive nature.

The survey will not include any questions of a sensitive nature.

3. Estimates of the hour burden of the collection of information.

The below table is based on an estimated number of completed surveys (25% response rate) of 382, at a maximum of 20 minutes per survey.

Expected Number of Respondents	Time Per Response	Total Annual Burden in Hours	Cost*
382 + 30 (pilot study) = 412	20 minutes	127.33	$4,788.81

*Cost burden calculated at $34.87 per hour (includes average hourly wage of $28.82 plus 20.99 % Railroad Retirement tax liability. Average hourly wage based on Bureau of Labor Statistics median wage for railroad employees; see http://www.bls.gov/oes/current/oes534099.htm).

3. Estimate of the total annual [non-hour] cost burden to respondents.

There will be no additional cost burden to respondents. There will be no need for respondents to keep any records associated with this data collection effort.

Estimates of annualized cost to the Federal government.

This is a one-time data collection and, while it should be done regularly to keep abreast of advancing technology, there is no commitment to reoccurrence. Based on GS-13 and current contractor labor rates and estimated supplies and services costs, the total cost to the Federal Government for this study is estimated at $193,543 over approximately 24 months, which amounts to an annual cost of approximately $96,771 per year.

Task	Cost per Task*
OMB Approval	$11,000
Stakeholder Panel	$23,600
Data Collection Plan	$6,000
Pilot Study	$15,100
Questionnaire Administration	$38,500
Clean, Compile, & Analyze Data	$41,600
Reporting	$35,600
Grand Total	$193,543

*At the time of this proposal, Costs per Task reflect an estimated 48.6% overall to be paid for contracted services. Actual Federal vs Contracted services will be determined at project launch pending human resource availability and Federal fiscal year funding disbursement for this study.

3. Explain the reasons for any program changes or adjustments.

This is a new, one-time collection. No adjustments are involved.

3. Publication of results of data collection.

The results of this study will be published in a written format by FRA. A summary of the results may also be presented at technical meetings, such as the annual meeting of the Transportation Research Board.

The planned project schedule, shown below, assumes that FRA will receive OMB approval for the study by the end of Fall 2018.

Activity	Date
OMB Approval	Fall 2018
Stakeholder Panel	Spring 2019 & 2020
Data Collection Plan	Fall 2018
Pilot Study	Spring–Summer 2019
Questionnaire Administration	Fall 2019–Winter 2019
Clean, Compile, & Analyze Data	Fall 2019–Summer 2020
Publication of Final Report	Fall 2020 (December 31, 2020)

3. Approval for not displaying the expiration date for OMB approval.

We are not seeking such approval.

3. Exception to the certification statement.

There are no exceptions to the certification statement.

References

Altschuld, J. W., & Kumar, D. D. (2005) Needs Assessment. In S. Mathison (Ed.), Encyclopedia of Evaluation (pp. 276–277).

American Association for Public Opinion Research, Cell Phone Task Force. (2010). New considerations for survey researchers when planning and conducting RDD telephone surveys in the U.S. with respondents reached via cell phone numbers, 2010. http://www.aapor.org/AAPOR_Main/media/MainSiteFiles/2010AAPORCellPhoneTFReport.pdf

Current Population Survey, Computer and Internet Use (2013), United States Census Bureau http://www.nber.org/cps/

Current Population Survey Internet Use (2010), United States Department of Commerce, National Telecommunications & Information Administration, http://www.ntia.doc.gov/data/CPS2010_Tables

Dillman, D. A. (2007). Mail and Internet Surveys: The tailored design method (2^nd ed.). Hoboken, NJ: John Wiley & Sons, Inc.

Dillman, D. A. (2011). Reconsidering Mail Survey Methods in an Internet World. American Statistical Association/American Association for Public Opinion Research Webinar slides, April 13, 2011. http://www.umb.edu/editor_uploads/images/u54/ASAAAPORwebinar4_11_11.pdf

Exploring the Digital Nation: America’s Emerging Online Experience (2013). National Telecommunications and Information Administration and Economics and Statistics Administration. http://www.ntia.doc.gov/report/2013/exploring-digital-nation-americas-emerging-online-experience

Federal Railroad Administration, Office of Research, Development & Technology, Human Factors Program, Key Functions, “Program Structure” para. 1 http://www.fra.dot.gov/Page/P0068

Federal Railroad Administration. (2013). Research and Development Strategic Plan, FY2013-FY2017 https://www.fra.dot.gov/Elib/Document/3184

File, T., & Ryan, C., “Computer and Internet Use in the United States: 2013,” American Community Survey Reports, ACS-28, U.S. Census Bureau, Washington, DC, 2014.

Information & Communication Technology Survey (2011), United States Census Bureau, http://www.census.gov/econ/ict/

Internet & American Life Project Tracking Surveys (2013). Pew Research Center, http://www.pewinternet.org/2013/06/10/tablet-ownership-2013/

Scriven, M. & Roth, J. (1990). Special feature: Needs assessment. Evaluation Practice, 11(2), 135-144.

Spence, DeYoung, & Feng. (2009). The technology profile inventory: Construction, validation, and application. Computers in human behavior, 25, 458-465

The Household Diary Study: Mail Use & Attitudes in FY 2012 (2013). United States Postal Service Headquarters, Washington, D.C. http://about.usps.com/current-initiatives/studying-americans-mail-use.htm

Verdegem, P., & De Marez, L. (2011). Rethinking determinants of ICT acceptance: Towards an integrated and comprehensive overview. Technovation, 31, 411-423. DOI: 10.1016/j.technovation.2011.02.004

Witkin, B. R., & Altschuld, J. W. (1995). Planning and conducting needs assessments: A practical guide. Thousand Oaks, CA: Sage.

10/24/2018 Page 8 of 8