Supporting Statement Part A for OMB Collection 1660-0039 (clean)

February 14, 2018

Supporting Statement for

Paperwork Reduction Act Submissions

OMB Control Number: 1660-0039

Title: National Fire Academy Long-Term Evaluation Form for Supervisors and National Fire Academy Long-Term Evaluation Form for Students/Trainees

Form Number(s): FEMA Form 078-0-2; FEMA Form 078-0-2A

General Instructions

A Supporting Statement, including the text of the notice to the public required by 5 CFR 1320.5(a)(i)(iv) and its actual or estimated date of publication in the Federal Register, must accompany each request for approval of a collection of information. The Supporting Statement must be prepared in the format described below, and must contain the information specified in Section A below. If an item is not applicable, provide a brief explanation. When Item 17 or the OMB Form 83-I is checked “Yes”, Section B of the Supporting Statement must be completed. OMB reserves the right to require the submission of additional information with respect to any request for approval.

Specific Instructions

A. Justification

1. Explain the circumstances that make the collection of information necessary.

Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information. Provide a detailed description of the nature and source of the information to be collected.

The National Fire Academy (NFA) is mandated under the Fire Prevention and Control Act of 1974 (Public Law 93-498 http://legcounsel.house.gov/Comps/FIREPREV.PDF) to provide training and education to the Nation’s fire service and emergency service personnel. The state-of-the-art programs offered by the NFA serve as models of excellence and state and local fire service agencies rely heavily on the curriculum to train their personnel. To maintain the quality of these training programs, it is critical that courses be evaluated after students have had the opportunity to apply the knowledge and skills gained from their training.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection. Provide a detailed description of: how the information will be shared, if applicable, and for what programmatic purpose.

Information collected with these evaluation forms enables U.S. Fire Administration (USFA) /NFA staff to monitor and recommend changes in course materials, individual subject selection criteria, and to make curriculum-wide reviews and assessments. The online electronic forms are processed through a Microsoft Structured Query Language (MS SQL) database and reports are made available electronically to USFA/NFA management and staff. Reports of longer term outcomes are annually posted online and disseminated in presentations by the NFA.

Below is a detailed explanation of how each form will be utilized:

1. FEMA Form 078-0-2, Long Term Evaluation for Supervisors – This form asks supervisors of students who have taken NFA training to judge if the training has improved the student’s job performance, contributed to their professional development, improved the department’s performance, led to a reduction in fire related risks within the community, and is the training worth recommending to others. Responses assist in maintaining quality of NFA training programs and respondent’s ability to apply the knowledge and skills gained in their department or community.

2. FEMA Form 078-0-2A, Long Term Evaluation for Students – This form asks students who have taken NFA training to judge if the training has improved the respondents job performance, contributed to their professional development, improved their department’s performance, led to a reduction in fire related risks within the community, and is the training worth recommending to others. Student’s report on specifics of training materials and experience of training diffusion. Responses assist in maintaining quality of NFA training programs and respondent’s ability to apply the knowledge and skills gained in their department or community.

Descriptive statistics obtained from each student and their supervisor, concerning training uses and effectiveness, are compiled for review by training and instructional systems specialists and USFA/NFA managers.

In particular, training specialists, instructional systems specialists and managers receive course specific reports which provide two types of study data. The two types of information include descriptive statistics, described below in section A, and narrative responses, described below in section B.

Section A. The first type of reported information is comprised of descriptive statistics, which deal with program effectiveness. The program effectiveness results provide NFA’s training specialists and management with information useful for consideration of curriculum and content revision.

The reports include percentage distributions of the following descriptive statistics:

(1) Supervisors who judge that the student’s training has:

• Improved the student’s job performance;

• Contributed to the student’s professional development;

• Improved the performance of the department;

• Led to a reduction in fire related risks within the community, and

• Has been worth recommending to others.

(2) Students who report specific applications of training materials, i.e., the proportion of students who indicate having applied specific training objectives.

These results permit an immediate assessment of NFA training in terms of both individual training classes and curriculum-wide applications. For example, NFA is able to assess what percent of training objectives are being applied by what percent of its students. In other words, NFA can report that “X” percent of students are actually applying “Y” percent of the training objectives presented in the training classes. Additionally, the results present a snapshot picture of which classes within the curriculum are having the widest application within the student’s local training organization.

(3) Students who judge that the student’s training has:

• Helped the department address all-hazard problems in community high-risk areas;

• Helped with policy development;

• Provided opportunities to collaborate and/or network with other NFA students.

(4) Students reporting their experience of training diffusion, i.e., the percent of students who have shared training knowledge and information:

• Formally in a training class;

• Informally in a group; or

• Informally, one on one.

Section B. The second type of information provided by the Long Term Evaluation (LTE) reports is based on narrative data and is prepared for training specialists and NFA managers. Narrative data vary but generally include comments provided by the students and supervisors about:

• Needed improvements in the NFA training:

• Descriptions of incidents in which students have applied NFA training: and

• Topics for future training classes.

In addition, special reports are run from LTE data for other internal and external audiences as requested.

There are several ways in which the narrative data are used. For example:

(1) Training specialists can determine the effectiveness of NFA training at advancing the skills of trainees and the resulting benefit to his/her organization,

(2) Instructional staff can determine appropriate methodologies and delivery modes for future NFA training, and

(3) U.S. Fire Administration/NFA management can evaluate the training on achievement of strategic goals that support the Agency’s performance measurement process.

The state-of-the-art programs offered by the NFA serve as models of excellence. State and local fire service agencies rely heavily on the curriculum to train their personnel. To maintain the high standards of these programs, it is critical that courses be evaluated after students have had the opportunity to apply the knowledge and skills gained.

This request includes information collection activity targeting NFA 2-day, 6-day, and 10-day training courses. This request seeks approval for two electronically based long-term evaluation (LTE) forms which will collect information from both course graduates and their respective supervisors.

The information to be collected addresses the student’s applications of specified training objectives upon returning to his or her local department or professional organization.

The data collection will be managed by NFA staff with expertise and experience in data collection, analysis and reporting.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

The survey activity includes both fully electronic and automated methods. Students volunteer to complete the LTE process while completing their end of course evaluations or by clicking on the LTE icon at a networked campus workstation. Automated email letters, which contain the secure LTE URL, https://apps.usfa.fema.gov/nfacourses/lte/signIn, are computer generated and sent to the signed-up students and their supervisors inviting them to complete the online form. For email messages which are returned undeliverable due to incorrect email addresses, physical letters containing the secure LTE URL, https://apps.usfa.fema.gov/nfacourses/lte/doSignIn, and a personal password are generated and mailed by US Postal Service. The objectives of the course set by the NFA Training Specialists determine when the survey activity occurs. This will range from 120 to 240 days following course participation. Participants have a 60-day turnaround time for completion.

The LTE can only be completed online and the data is immediately stored in a MS SQL database. Standard reports are generated directly from the MS SQL database and targeted searches can be specified and data downloaded for further analysis using MS Excel macros and SPSS syntax files.

The electronic format of the LTE reduces the paper burden of students and supervisors. All students are requested to sign-up both themselves and their supervisor for the LTE when completing the End of Course (EOC) evaluation during their time at the NFA. Those students and supervisors receive the LTE link as scheduled.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above. 

This information is not collected in any other form; therefore is not duplicated elsewhere.

5. If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize.

This information collection does not have an impact on small businesses or other small entities.

6. Describe the consequence to Federal/FEMA program or policy activities if the collection of information is not conducted, or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

If the data collection is not conducted, the USFA/NFA will not be able to determine its effectiveness in carrying out its mission to provide effective training and education for the Nation’s fire and emergency response community.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

1. Requiring respondents to report information to the agency more often than quarterly.

 (b) Requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it.

3. Requiring respondents to submit more than an original and two

copies of any document.

3. Requiring respondents to retain records, other than health,

medical, government contract, grant-in-aid, or tax records for more than three years.

3. In connection with a statistical survey, that is not designed to

produce valid and reliable results that can be generalized to the universe of study.

 (f) Requiring the use of a statistical data classification that has not

been reviewed and approved by OMB.

 (g) That includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use.

 (h) Requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.

The special circumstances contained in item 7 of the supporting statement are not applicable to this information collection.

8. Federal Register Notice: 

 a. Provide a copy and identify the date and page number of publication in the Federal Register of the agency’s notice soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

A 60-day Federal Register Notice inviting public comments was published on October 24, 2017, 82 FR 49225. FEMA received 36 anonymous public comments that were not relevant to the information collection.

A 30-day Federal Register Notice inviting public comments was published on February 7, 2018, 83 FR 5448. No comments were received. See attached copy of the published notice included in this package.

 b. Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

Contracted personnel provide expert assistance only on the operation of the data collection software/system and will be consulted in the availability and operation of such.

c. Describe consultations with representatives of those from whom information is to be obtained or those who must compile records. Consultation should occur at least once every three years, even if the collection of information activities is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

The NFA consults with its residential student population on a regular basis regarding course evaluations. The Superintendent holds a “Superintendent’s Lunch” with class representatives during every resident class delivery cycle. This luncheon provides the opportunity for the Superintendent and class representatives to discuss issues concerning training development, course delivery and course evaluation. The Superintendent also visits each class during every delivery cycle and provides an open forum for any issues the students and instructors want to discuss. Therefore, consultations occur at least once every three years.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

FEMA does not provide payments or gifts to respondents in exchange for a benefit sought.

10. Describe any assurance of confidentiality provided to respondents. Present the basis for the assurance in statute, regulation, or agency policy.

A Privacy Threshold Analysis (PTA) was approved by the DHS Privacy Office on November 14, 2017.

The Privacy Impact Assessment (PIA) is covered under the DHS/ALL/PIA-006 DHS General Contacts List, approved by DHS on June 15, 2007. Additionally, the System of Records Notice (SORN) for this collection is DHS/ALL-002 DHS Mailing and Other Lists, November 25, 2008, 73 FR 71659.

There are no assurances of confidentiality provided to the respondents for this information collection.

11. Provide additional justification for any question of a sensitive nature (such as sexual behavior and attitudes, religious beliefs and other matters that are commonly considered private). This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

There are no questions of a sensitive nature.

 12. Provide estimates of the hour burden of the collection of information. The statement should:

 a. Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated for each collection instrument (separately list each instrument and describe information as requested). Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desired. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.

It is anticipated that 1,500 students will complete FEMA Form 078-0-2A. Each respondent will only complete the form once and each response will require 0.17 hours to complete the form. The total annual hour burden is 1,500 x 0.17 hours = 255 annual hours.

It is anticipated that 1,500 supervisors will complete FEMA Form 078-0-2. Each respondent will only complete the form once and each response will require 0.1 hour to complete the form. The total annual hour burden is 1,500 x 0.1 hours = 150 annual hours.

 b. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.

c. Provide an estimate of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. NOTE: The wage-rate category for each respondent must be multiplied by 1.4 and this total should be entered in the cell for “Avg. Hourly Wage Rate”. The cost to the respondents of contracting out or paying outside parties for information collection activities should not be included here. Instead this cost should be included in Item 13.

Estimated Annualized Burden Hours and Costs
Type of Respondent	Form Name / Form Number	No. of Respon-dents	No. of Respon-ses per Respon-dent	Total No. of Responses	Avg. Burden per Response (in hours)	Total Annual Burden (in hours)	Avg. Hourly Wage Rate	Total Annual Respondent Cost
State, Local or Tribal Government	NFA Long Term Evaluation for Students FEMA Form 078-0-2A	1,500	1	1,500	10.2 minutes (0.17)	255	$35.46	$9,042.30
State, Local or Tribal Government	NFA Long Term Evaluation for Supervisors/ FEMA Form 078-0-2	1,500	1	1,500	6 minutes (0.10)	150	$54.08	$8,112.00
Total		3,000		3,000		405		$17,154.30

Note: The “Avg. Hourly Wage Rate” for each respondent includes a 1.46 multiplier to reflect a fully-loaded wage rate.

According to the U.S. Department of Labor, Bureau of Labor Statistics (BLS) website ¹ the wage rate category for fire fighters is estimated to be $24.29 per hour, which is multiplied by a wage rate multiplier of 1.46 ^{2 3}to calculate a fully-loaded wage rate of $35.46 per hour. ($24.29 x 1.46 = $34.84) Therefore, the estimated burden hour cost to respondents who are Fire Fighters is estimated to be $9,042.30 annually and 255 burden hours. (255 hours x $35.26 per hour = $9,042.30)

According to the BLS website ⁴ the wage rate category for First-line Supervisors of Fire Fighting and Prevention Workers is estimated to be $37.04 per hour, which is multiplied by a wage rate multiplier of 1.46 to calculate a fully-loaded wage rate of $54.08 per hour. ($37.04 x 1.46 = $54.08) Therefore, the estimated burden hour cost to respondents who are First-line Supervisors of Fire Fighting and Prevention Workers is estimated to be $8,112 annually and 150 burden hours. (150 hours x $54.08 per hour = $8,122.00)

13. Provide an estimate of the total annual cost burden to respondents or recordkeepers resulting from the collection of information. The cost of purchasing or contracting out information collection services should be a part of this cost burden estimate. (Do not include the cost of any hour burden shown in Items 12 and 14.)

The cost estimates should be split into two components:

a. Operation and Maintenance and purchase of services component. These estimates should take into account cost associated with generating, maintaining, and disclosing or providing information. Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred.

b. Capital and Start-up-Cost should include, among other items, preparations for collecting information such as purchasing computers and software, monitoring sampling, drilling and testing equipment, and record storage facilities.

There are no record keeping, capital, start-up or maintenance costs associated with this information collection.

There are no record keeping, capital, start-up or maintenance costs associated with this information collection.

14. Provide estimates of annualized cost to the federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing and support staff), and any other expense that would have been incurred without this collection of information. You may also aggregate cost estimates for Items 12, 13, and 14 in a single table.

* Note: The “Salary Rate” includes a 1.46 multiplier to reflect a fully-loaded wage rate.

 15. Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-I in a narrative form. Present the itemized changes in hour burden and cost burden according to program changes or adjustments in Table 5. Denote a program increase as a positive number, and a program decrease as a negative number.

A "Program increase" is an additional burden resulting from a federal government regulatory action or directive. (e.g., an increase in sample size or coverage, amount of information, reporting frequency, or expanded use of an existing form). This also includes previously in-use and unapproved information collections discovered during the ICB process, or during the fiscal year, which will be in use during the next fiscal year.

A "Program decrease", is a reduction in burden because of: (1) the discontinuation of an information collection; or (2) a change in an existing information collection by a Federal agency (e.g., the use of sampling (or smaller samples), a decrease in the amount of information requested (fewer questions), or a decrease in reporting frequency).

"Adjustment" denotes a change in burden hours due to factors over which the government has no control, such as population growth, or in factors which do not affect what information the government collects or changes in the methods used to estimate burden or correction of errors in burden estimates.

Itemized Changes in Annual Burden Hours
Data collection Activity/Instrument	Program Change (hours currently on OMB Inventory)	Program Change (New)	Difference	Adjustment (hours currently on OMB Inventory)	Adjustment (New)	Difference
NFA Long Term Evaluation Students / Trainees / FEMA Form 078-0-2A				255	255	0
NFA Long Term Evaluation Supervisors / FEMA Form 078-0-2				150	150	0
Total(s)				405	405	0

Explain:

There are no changes to the annual hour burden previously reported and there has been no change to the information being collected. However, there have been some changes to FEMA Form 078-0-2A and FEMA Form 078-0-2. One question was added to each form to better reflect and measure the NFA’s progress toward the USFA’s goals. Two questions were removed from the student/trainee form that are no longer relevant to NFA’s data collection.

Itemized Changes in Annual Cost Burden

Explain:

There is no cost burden for this collection.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

The Long Term Evaluation Report is published annually for each Fiscal Year on the NFA Web site. The annual LTE report contains graphic representations of (1) percent agreement by students and supervisors with course satisfaction statements, (2) the relative frequency and utility-rating of training experiences at the NFA and other training locations, (3) estimates for the amount of training diffusion on the job and to work colleagues, and (4) relative frequency of various suggestions for improvements and new training courses. These results are system generated with minimal demand on the time (8 hours) of an NFA staff member.

17. If seeking approval not to display the expiration date for OMB approval of the information collection, explain reasons that display would be inappropriate.

FEMA will display the expiration date for OMB approval of this information collection.



18. Explain each exception to the certification statement identified in Item 19 “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-I.

 FEMA does not request an exception to the certification of this information collection.

B. Collections of Information Employing Statistical Methods.



There is no statistical methodology involved in this collection.

1 May 2016 National Occupational Employment and Wage Rates, National File (xls), Firefighters (OCC Code: 33-2010, Average, Column Title: H_Mean). Accessed and downloaded October 5, 2017.

https://www.bls.gov/oes/tables.htm

2 Bureau of Labor Statistics, Employer Costs for Employee Compensation, Table 1. Employer costs per hour worked for employee compensation and costs as a percent of total compensation: Civilian workers, by major occupational and industry group, June 2017. Accessed and downloaded October 5, 2017.

https://www.bls.gov/news.release/archives/ecec_09082017.htm

3 The per hour benefits multiplier is calculated by dividing total compensation for all workers ($35.28, September 8, 2017) by wages and salaries for all workers ($24.10, September 8, 2017), which yields a per hour benefits multiplier. For September 8, 2017, the multiplier is 1.46. ($35.28 ÷ $24.10 = 1.463900). Fully-loaded wage rates are calculated by multiplying the per hour benefits multiplier by the applicable wage rate from the applicable National Occupational Employment and Wage Rates report. (1.46 per hour benefits multiplier x hourly wage rate = fully-loaded hourly wage)

4 May 2016 National Occupational Employment and Wage Rates, National File (xls), First-Line Supervisory of Fire Fighting and Prevention Workers (OCC Code: 33-1020, Average, Column Title: H_Mean). Accessed and downloaded October 5, 2017.

https://www.bls.gov/oes/tables.htm

13