1660-0039 - Supporting Statement A - 2024 11 08 clean

November 8, 2024

Supporting Statement for
Paperwork Reduction Act Submissions

OMB Control Number: 1660-0039

Title: National Fire Academy Long-Term Evaluation Form for Supervisors and National Fire Academy Long-Term Evaluation Form for Students/Trainees

Form Number(s):

1. FEMA Form FF-USFA-FY-22-118 (formerly 078-0-2), National Fire Academy Long Term Evaluation for Supervisors;

2. FEMA Form FF-USFA-FY-22-119 (formerly 078-0-2A), National Fire Academy Long Term Evaluation for Students

General Instructions

A Supporting Statement, including the text of the notice to the public required by 5 CFR 1320.5(a)(1)(iv) and its actual or estimated date of publication in the Federal Register, must accompany each request for approval of a collection of information. The Supporting Statement must be prepared in the format described below, and must contain the information specified in Section A below. If an item is not applicable, provide a brief explanation. When Item 17 or the OMB Form 83-I is checked “Yes”, Section B of the Supporting Statement must be completed. OMB reserves the right to require the submission of additional information with respect to any request for approval.

Specific Instructions

A. Justification

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information. Provide a detailed description of the nature and source of the information to be collected.

The National Fire Academy (NFA) is mandated under the Fire Prevention and Control Act of 1974 (Public Law 93-498) to provide training and education to the Nation’s fire service and emergency service personnel. The state-of-the-art programs offered by the NFA serve as models of excellence and state and local fire service agencies rely heavily on the curriculum to train their personnel. To maintain the quality of these training programs, it is critical that courses be evaluated after students have had the opportunity to apply the knowledge and skills gained from their training.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection. Provide a detailed description of: how the information will be shared, if applicable, and for what programmatic purpose.

Information collected with these evaluation forms enables U.S. Fire Administration (USFA)/NFA staff to monitor and recommend changes in course materials, individual subject selection criteria, and to make curriculum-wide reviews and assessments. The online electronic forms are processed through a Microsoft Structured Query Language (MS SQL) database and reports are made available electronically to USFA/NFA management and staff. Reports of longer-term outcomes are annually posted online and disseminated in presentations by the NFA.

Below is a detailed explanation of how each form will be utilized:

FEMA Form FF-USFA-FY-22-118 (formerly 078-0-2), Long Term Evaluation for Supervisors: This form asks supervisors of students who have taken NFA training to judge if the training has improved the student’s job performance, contributed to their professional development, improved the department’s performance, led to a reduction in fire related risks within the community, and is the training worth recommending to others. Responses assist in maintaining quality of NFA training programs and respondent’s ability to apply the knowledge and skills gained in their department or community.

FEMA Form FF-USFA-FY-22-119 (formerly 078-0-2A), Long Term Evaluation for Students: This form asks students who have taken NFA training to judge if the training has improved the respondents job performance, contributed to their professional development, improved their department’s performance, led to a reduction in fire related risks within the community, and is the training worth recommending to others. Student’s report on specifics of training materials and experience of training diffusion. Responses assist in maintaining quality of NFA training programs and respondent’s ability to apply the knowledge and skills gained in their department or community.

Descriptive statistics obtained from each student and their supervisor, concerning training uses and effectiveness, are compiled for review by training and instructional systems specialists and USFA/NFA managers.

In particular, training specialists, instructional systems specialists and managers receive course specific reports which provide two types of study data. The two types of information include descriptive statistics, described below in section A, and narrative responses, described below in section B.

Section A. The first type of reported information is comprised of descriptive statistics, which deal with program effectiveness. The program effectiveness results provide NFA’s training specialists and management with information useful for consideration of curriculum and content revision.

The reports include percentage distributions of the following descriptive statistics:

1. Supervisors who judge that the student’s training has:

• Improved the student’s job performance;

• Contributed to the student’s professional development;

• Improved the performance of the department;

• Led to a reduction in fire related risks within the community, and

• Has been worth recommending to others.

2. Students who report specific applications of training materials.

These results permit an assessment of NFA training in terms of both individual training classes and curriculum-wide applications. For example, NFA is able to assess what percent of students are applying training information learned to their jobs. Additionally, the results present a snapshot picture of which classes within the curriculum are having the widest application within the student’s local training organization.

3. Students who judge that the student’s training has:

• Helped the department address all-hazard problems in community high-risk areas;

• Helped with policy development;

• Provided opportunities to collaborate and/or network with other NFA students.

4. Students reporting their experience of training diffusion, i.e., the percent of students who have shared training knowledge and information:

• Formally in a training class;

• Informally in a group; or

• Informally, one on one.

Section B. The second type of information provided by the Long-Term Evaluation (LTE) reports is based on narrative data and is prepared for training specialists and NFA managers. Narrative data varies but generally include comments provided by the students and supervisors about:

• Needed improvements in the NFA training:

• Descriptions of incidents in which students have applied NFA training: and

• Topics for future training classes.

In addition, special reports are run from LTE data for other internal and external audiences as requested.

There are several ways in which the narrative data are used. For example:

1. Training specialists can determine the effectiveness of NFA training at advancing the skills of trainees and the resulting benefit to his/her organization,

2. Instructional staff can determine appropriate methodologies and delivery modes for future NFA training, and

3. U.S. Fire Administration/NFA management can evaluate the training on achievement of strategic goals that support the Agency’s performance measurement process.

The state-of-the-art programs offered by the NFA serve as models of excellence. State and local fire service agencies rely heavily on the curriculum to train their personnel. To maintain the high standards of these programs, it is critical that courses be evaluated after students have had the opportunity to apply the knowledge and skills gained.

This request includes information collection activity targeting NFA two-day, six-day, and ten-day training courses. This request seeks approval for two electronically based long-term evaluation (LTE) forms which will collect information from both course graduates and their respective supervisors.

The information to be collected addresses the student’s applications of specified training objectives upon returning to his or her local department or professional organization.

The data collection will be managed by NFA staff with expertise and experience in data collection, analysis, and reporting.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

The survey activity includes both fully electronic and automated methods. Students and their supervisor names and emails are retrieved from electronic NFA class rosters. Automated email letters, which contain the secure LTE URL https://apps.usfa.fema.gov/nfacourses/lte/signIn, are computer generated and sent to the students and their supervisors inviting them to complete the online form, https://apps.usfa.fema.gov/nfacourses/lte/doSignIn. Each student and supervisor is provided with an individual generated login ID and password in the email letter.

NFA Training Specialists determine when the survey activity occurs. This will range from 120 to 240 days following course participation. Participants have a 60-day turnaround time for completion.

The LTE can only be completed online and the data is immediately stored in an MS SQL database. Standard reports are generated directly from the MS SQL database and targeted searches can be specified and data downloaded for further analysis using MS Excel statistical software.

The electronic format of the LTE reduces the paper burden of students and supervisors.

NFA conducted a usability test on both the student form and the supervisor form to determine the ease of use of the web-based system, the readability of each form, and to estimate the burden for completing each form. Five NFA staff members, who agreed to participant in the usability test, were given the link to the forms and user login credentials to access for each form. These participants were aware that these forms were used by NFA to gather data from students and supervisors, but none of them had ever seen the actual forms prior to the test.

Participants were asked to complete one form on their computer and one on a personal hand-held device, such as a cell phone. They were assigned which form to complete on which type of device. They were asked to record the time it took to compete each form and to note any difficulties they had or questions that came up as they completed the forms. They were also asked to provide feedback on the web-based system and the readability of each form. The participants noted that the system was easy to navigate, and the questions were easy to understand and answer. No difference was noted between the form completed on the computer and the one completed on the cell phone. The average recorded completion times for each form matched the times already estimated on the forms. As result, there is no change to this collection at this time.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

This information is not collected in any other form; therefore is not duplicated elsewhere.

5. If the collection of information impacts businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize.

This information collection does not have an impact on small businesses or other small entities. The collection is voluntary and does not have an impact on small businesses or other small entities.

6. Describe the consequence to Federal/FEMA program or policy activities if the collection of information is not conducted, or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

If the data collection is not conducted, the USFA/NFA will not be able to determine its effectiveness in carrying out its mission to provide effective training and education for the Nation’s fire and emergency response community.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner (See 5 CFR 1320.5(d)(2)):

1. Requiring respondents to report information to the agency more often than quarterly.

This information collection does not require respondents to report information more than quarterly.

2. Requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it.

This information collection does not require respondents to prepare a written response in fewer than 30 days after receipt of it.

3. Requiring respondents to submit more than an original and two copies of any document.

This information collection does not require respondents to submit more than an original and two copies of any document.

4. Requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years.

This information collection does not require respondents to retain records (other than health, medical, government contract, grant-in-aid, or tax records) for more than three years.

5. In connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study.

This information collection does not include a statistical survey.

6. Requiring the use of a statistical data classification that has not been reviewed and approved by OMB.

This information collection does not use a statistical data classification that has not been reviewed and approved by OMB.

7. That includes a pledge of confidentiality that is not supported by authority established in statue or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use.

This information collection does not include a pledge of confidentiality that is not supported by established authorities or policies.

8. Requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.

This information collection does not require respondents to submit trade secrets or other confidential information.

8. Federal Register Notice:

1. Provide a copy and identify the date and page number of publication in the Federal Register of the agency’s notice soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

A 60-day Federal Register Notice inviting public comments was published on July 22, 2024, at 89 FR 59128. No comments were received.

A 30-day Federal Register Notice inviting public comments was published on November 8, 2024, at 89 FR 88799. The public comment period is open until December 9, 2024.

2. Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

Contracted personnel provide expert assistance only on the operation of the data collection software/system and will be consulted in the availability and operation of such.

3. Describe consultations with representatives of those from whom information is to be obtained or those who must compile records. Consultation should occur at least once every three years, even if the collection of information activities is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

The NFA consults with its residential student population on a regular basis regarding course evaluations. The Superintendent visits each class during every delivery cycle and provides an open forum for any issues the students and instructors want to discuss. Therefore, consultations occur at least once every three years.

9. Explain any decision to provide any payments or gift to respondents, other than remuneration of contractors or grantees.

FEMA does not provide payments or gifts to respondents in exchange for a benefit sought.

10. Describe any assurance of confidentiality provided to respondents. Present the basis for the assurance in statute, regulation, or agency policy.

A Privacy Threshold Analysis (PTA) was approved by the DHS Privacy Office on January 18, 2024.

The Privacy Impact Assessment (PIA) is covered under the DHS/ALL/PIA-006 DHS General Contacts List, approved by DHS on April 25, 2019. Additionally, the System of Records Notice (SORN) for this collection is DHS/ALL-002 DHS Mailing and Other Lists, November 25, 2008, 73 FR 71659.

There are no assurances of confidentiality provided to the respondents for this information collection.

11. Provide additional justification for any question of a sensitive nature ( such as sexual behavior and attitudes, religious beliefs and other matters that are commonly considered private). This should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

There are no questions of a sensitive nature.

12. Provide estimates of the hour burden of the collection of information. The statement should:

1. Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated for each collection instrument (separately list each instrument and describe information as requested). Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consolation with a sample (fewer than 10) of potential respondents is desired. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.

FEMA Form FF-USFA-FY-22-118 (formerly 078-0-2), Long Term Evaluation for Supervisors: is estimated to have 1500 respondents multiplied by 1 response(s) per year for 1500 total annual responses (1500 x 1 = 1500). It is estimated that each response will 0.1 burden hours to complete, therefore 1500 responses times 0.1 hours equals 150 total annual burden hours (1500 x 0.1 =150).

FEMA Form FF-USFA-FY-22-119 (formerly 078-0-2A), Long Term Evaluation for Students: is estimated to have 1500 respondents times 1 response(s) per year for 1500 total annual responses (1500 x 1 = 1500). It is estimated that each response will 0.17 burden hours to complete, therefore 1500 responses times 0.17 hours equals 255 total annual burden hours (1500 x 0.17 =255).

After conducting usability testing there is no change to the burden hours as listed in the OMB inventory.

2. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.

Please see our response for 12a above and 12c below.

3. Provide an estimate of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. NOTE: The wage-rate category for each respondent must be multiplied by 1.45 (1.61 for State and local government employees)¹ and this total should be entered in the cell for “Avg. Hourly Wage Rate.” The cost to the respondents of contracting out to paying outside parties for information collection activities should not be included here. Instead this cost should be included in Item 13

Estimated Annualized Burden Hours and Costs
Type of Respondent	Form Name / Form Number	No. of Respondents	No. of Responses per Respondent	Total No. of Responses	Avg. Burden per Response (in hours)	Total Annual Burden (in hours)	Avg. Hourly Wage Rate	Total Annual Respondent Cost
State, Local or Tribal Government	NFA Long Term Evaluation for Supervisors FF-USFA-FY-22-118	1,500	1	1,500	0.10	150	$65.70	$9,855
State, Local or Tribal Government	NFA Long Term Evaluation for Students FF-USFA-FY-22-119	1,500	1	1,500	0.17	255	$47.48	$12,107
Total		3,000		3,000		405		$21,962

Note: The “Avg. Hourly Wage Rate” for each respondent include a wage multiplier to reflect a fully-loaded wage rate.

“Type of Respondent should be entered exactly as chosen in Question 3 of the OMB Form 83-I.

Instruction for Wage-rate category multiplier: Take each non-loaded “Avg. Hourly Wage Rate” from the BLS website table and multiply that number by 1.61. For example, a non-loaded BLS table wage rate of $42.51 would be multiplied by 1.61, and the entry for the “Avg. Hourly Wage Rate” would be $68.44.

According to the U.S. Department of Labor, Bureau of Labor Statistics website (www.bls.gov) the May 2023 wage rate category for SOC 33-1021 First-Line Supervisors of Firefighting and Prevention Workers working for State governments is estimated to be $40.81 per hour.² Including the wage rate multiplier of 1.61, the fully loaded wage rate is $65.70. Therefore, the estimated burden hour cost to respondents who are First-Line Supervisors of Firefighting and Prevention Workers is estimated to be 150 hours x 65.70 per hour = $9,855 annually.

According to the U.S. Department of Labor, Bureau of Labor Statistics website (www.bls.gov) the May 2023 wage rate category for SOC 33-2000 Firefighters is estimated to be $29.49 per hour.³ Including the wage rate multiplier of 1.61, the fully loaded wage rate is $47.48. Therefore, the estimated burden hour cost to respondents who are Firefighters is estimated to be 255 hours x 47.48 per hour = $12,107 annually.

13. Provide an estimate of the total annual cost burden to respondents or recordkeepers resulting from the collection of information. The cost of purchasing or contracting out information collection services should be a part of this cost burden estimate. (Do not include the cost of any hour burden shown in Items 12 and 14.)

The cost estimates should be split into two components:

1. Operation and Maintenance and purchase of services component. These estimates should take into account cost associated with generating, maintaining, and disclosing or providing information. Include descriptions of methods used to estimate major cost factors including systems and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred.

There are no operation or maintenance costs associated with this information collection.

2. Capital and Start-Up Cost should include, among other items, preparations for collecting information such as purchasing computers and software, monitoring sampling, drilling and testing equipment, and record storge facilities.

There are no capital or start-up costs associated with this information collection.

14. Provide estimates of annualized cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing and support staff), and any other expense that would have been incurred without this collection of information. You may also aggregate cost estimates for Items 12, 13, and 14 in a single table.

Annual Cost to the Federal Government
Item	Cost ($)
Contract Costs: Funding the USFA Web Farm Services used in maintaining the application.	$40,000
Staff Salaries: One GS-12/Step 5 spending approximately 15% of time annually on this data collection (15% )$112,425 x 1.45= $24,452	$24,452
Facilities [cost for renting, overhead, etc. for data collection activity]	$0
Computer Hardware and Software [cost of equipment annual lifecycle]	$0
Equipment Maintenance [cost of annual maintenance/service agreements for equipment]	$0
Travel (not to exceed)	$0
Total	$64,452
1 Office of Personnel Management 2024 Pay and Leave Tables for the Washington-Baltimore-Arlington, DC-MD-VA-WV-PA locality. Available online at https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2024/DCB.pdf. 2 Wage rate includes a 1.45 multiplier to reflect the fully-loaded wage rate.

15. Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of OMB Form 83-I in a narrative form. Present the itemized changes in hour burden and cost burden according to program changes or adjustments in Table 5. Denote a program increase as a positive number, and a program decrease as a negative number.

A “Program increase” is an additional burden resulting from a Federal Government regulation action or directive (e.g., an increase in sample size or coverage, amount of information, reporting frequency, or expanded use of an existing form). This also includes previously in-use and unapproved information collection discovered during the ICB process, or during the fiscal year, which will be in use during the next fiscal year.

A “Program decrease” is a reduction in burden because of: (1) the discontinuation of an information collection, or (2) a change in an existing information collection by a Federal Agency (e.g., the use of sampling (or smaller samples), a decrease in the amount of information requested (fewer questions), or a decrease in reporting frequency).

An “Adjustment” denotes a change in burden hours due to factors over which the government has no control, such as population growth, or in factors which do not affect what information the government collects or changes in the methods used to estimate burden or correction of errors in burden estimates.

Explain: There are no changes to the annual hour burden previously reported and there has been no change to the information being collected. No changes were made to the forms FF-USFA-FY-22-118 and FF-USFA-FY-22-119.

Explain: The annual cost burden for this collection was adjusted to incorporate updated wage rates based on the BLS May 2023 Occupational Employment and Wage Estimates for OCC Code 33-2000 code used was for Firefighter and Prevention Workers and OCC -33-1021 First-Line Supervisors of Firefighting and Prevention Workers. These adjustments increased the annual cost from $20,741 to $27,319.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

The LTE Report is published annually for each Fiscal Year on the NFA Web site. The annual LTE report contains graphic representations of (1) percent agreement by students and supervisors with course satisfaction statements, (2) the relative frequency and utility-rating of training experiences at the NFA and other training locations, (3) estimates for the amount of training diffusion on the job and to work colleagues, and (4) relative frequency of various suggestions for improvements and new training courses. These results are system generated with minimal demand on the time (eight hours) of an NFA staff member.

17. If seeking approval not to display the expiration date for OMB approval of the information collection, explain reasons that display would be inappropriate.

FEMA will display the expiration date for OMB approval of this information collection.

18. Explain each exception to the certification state identified in Item 19 “Certification for Paperwork Reduction Act Submission,” of OMB Form 83-I.

FEMA does not request an exception to the certification of this information collection.

1 Bureau of Labor Statistics, Employer Costs for Employee Compensation, Table 1.  Available at https://www.bls.gov/news.release/archives/ecec_03132024.pdf. Accessed March 13, 2024. The national wage multiplier is calculated by dividing total compensation for all workers of $45.42 by wages and salaries for all workers of $31.29 per hour yielding a benefits multiplier of approximately 1.45. For State and local government employees the wage multiplier is calculated by dividing total compensation for State and local government workers of $60.56 by Wages and salaries for State and local government workers of $37.53 per hour yielding a benefits multiplier of approximately 1.61.

2 Information on the mean wage rate from the U.S. Department of Labor, Bureau of Labor Statistics is available online at: https://www.bls.gov/oes/2023/may/oes_nat.htm

3 Information on the mean wage rate from the U.S. Department of Labor, Bureau of Labor Statistics is available online at: https://www.bls.gov/oes/2023/may/oes_nat.htm

Page 15 of 15