1660-0132 NTED Supporting Statement A_UPDATE

July 24, 2021

Supporting Statement for

Paperwork Reduction Act Submissions

OMB Control Number: 1660 - 0132

Title: Consolidated FEMA-National Training and Education Division (NTED) Level 3 Training Evaluation Forms

Form Number(s):

FEMA Form 016-0-2, Post Course Assessment National Training & Education Division, Training Partners Program (TPP); FEMA Form 092-0-2A, PER-220 Field Force Operations (FFO) Post-Graduate Questionnaire for Students; FEMA Form 092-0-2B, PER-220 Field Force Operations (FFO) Post-Graduate Training Evaluation for Supervisors; and FEMA Form 519-0-1, Emergency Management Institute (EMI): FEMA Follow-up Evaluation Survey.

General Instructions

A Supporting Statement, including the text of the notice to the public required by 5 CFR 1320.5(a)(i)(iv) and its actual or estimated date of publication in the Federal Register, must accompany each request for approval of a collection of information. The Supporting Statement must be prepared in the format described below and must contain the information specified in Section A below. If an item is not applicable, provide a brief explanation. When Item 17 or the OMB Form 83-I is checked “Yes”, Section B of the Supporting Statement must be completed. OMB reserves the right to require the submission of additional information with respect to any request for approval.

Specific Instructions

A. Justification

1. Explain the circumstances that make the collection of information necessary.

Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information. Provide a detailed description of the nature and source of the information to be collected.

TPP, CDP and EMI are geographically separated components of FEMA’s National Preparedness Directorate’s, National Training and Education Division (NTED). As such, all three organizations are part of the National Training and Education System (NTES) with unique training missions for specific target audiences and are responsible for supporting and enabling national preparedness Core Capability, Community Lifelines, and Recovery Sector Preparedness performance outcomes described in Presidential Policy Directive (PPD) 8 National Preparedness, the National Preparedness Goal (NPG), the National Preparedness System (NPS), and 2018-2022 FEMA Strategic Plan objectives 1.3, 2.1, and 3.3.

Like other Federal training organizations the TPP, CDP and EMI utilize the Kirkpatrick Training Evaluation Model Level 3 survey results as one of several training evaluation tools to inform component program managers and leadership with metrics that monitor and report organizational performance and compliance with the Government Performance, Results and Modernization Act of 2010. Section 1115 (A) Federal Gov. and (B) Agency Performance Plans (6) requires “…a balanced set of performance indicators to be used in measuring or assessing progress toward each performance goal, including, as appropriate, customer service, efficiency, output, and outcome indicators.” Due to the qualitative nature of participants’ feedback obtained from this survey measuring “impact or changes in course graduates work performance, their occupations or organizational performance behavior” the results, with other data, are used to monitor performance of TPP, CDP and EMI training program and NTED Training & Education portfolio performance. TPP and CDP are members of the National Domestic Preparedness Consortium (NDPC). Per 6 USC § 1102 (c) Duties the NDPC shall identify, develop, test [assess], and deliver training to State, local, and tribal emergency response providers, provide on-site and mobile training at the performance and management and planning levels, and facilitate the delivery of training by the training partners of the Department. (See Appendix A to this Supporting Statement, “Overview of Evaluation Plan”.)

6 U.S.C. § 748 directs the FEMA Administrator to implement (a) National Training Program (1) In general-“…the Administrator, in coordination with heads of appropriate Federal agencies, the National Council on Disability, and National Advisory Council, shall carry out a national training program to implement the national preparedness goal, NIMS, National Response Plan and other related plans and strategies. (2) Training Partners. – In developing and implementing the national training program, the Administrator shall- (A) Work with government training facilities, academic institutions, private sector organizations, and other entities that provide specialized, state-of-the-art training for emergency managers or emergency response providers; and (B) Utilize, as appropriate, training courses provided by community colleges, State and local public safety academies, State and private universities and other facilities.

The Level 3 data collected from course graduates 3 – 6 months following training provides TPP, CDP and EMI with information and data how training knowledge, skill and attitudes (KSA’s) gained from training transfer to the student’s job and organization and if the training was applied to the job, an exercise, or during an operational mission. Note: The number of TPP CTG training providers may fluctuate, increase or decrease based on FY grant funding levels approved by Congress.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection. Provide a detailed description of: how the information will be shared, if applicable, and for what programmatic purpose.

As stated in preceding paragraph; TPP, CDP, and EMI all utilize the Kirkpatrick Four Level Training Evaluation Model to assess training effectiveness. Donald Kirkpatrick developed the four-level model of training evaluation: Level 1 evaluation captures the students’ perception or reaction to training; Level 2 completes a pre-post comparison of learning via tests/examinations and performance assessment checklists of the student learning (change in behavior). Level 3 evaluation instruments capture how the training transfers to the students’ work environment and their particular job; and Level 4 captures information about the return on key stakeholder expectations of the course or training program. The data collected by these training evaluation forms is shared with representatives from each course/program’s respective target audience during three-year course/program reviews. All three organizations analyze the data to determine what revisions are needed in curriculum, delivery and to elicit inputs to improve training while providing documentation as to how training was applied, and how it impacts the individual and organization. The supporting statements immediately below are for each NTED component:

1. TPP: “Post Course Assessment Form NTED-TPP.” The TPP Level 3 Post Course Assessment Form is new and has been assigned FEMA Form # 016-0-2. Like CDP and EMI the TPP Branch developed a Level 3 instrument that contains five common questions or data elements also used in the CDP and EMI instruments in addition with other quantitative and qualitative questions to obtain information from course graduates approx. 3 – 6 months following training. Each TPP partner is responsible for sending graduates of their courses an email containing instruction on where and how to download, complete, and successfully return their level 3 survey. As each training partner receives and review individual responses for each course, they create a batch file and enter data into the TPP Registration Evaluation System (RES) database. All TPP component organizations share training data and information collected with representatives from their respective audiences during scheduled three-year course reviews.

2. CDP: Post Graduate Evaluation Form for Students, FEMA Form 092-0-2(A): Level 3 evaluations collect data on training transfer to the student’s workplace. This Post Graduate Evaluation provides information and data to measure the program effectiveness. The instrument is a product and process output and outcome measure used to capture the degree to which the course material effects the performance of each student in their place of employment or operations. The data is used to assess the validity and utility of course material as it relates to the actual performance of duties. CDP and other NTED components analyze the data for trends and gaps to determine what revisions, if any, are needed in curriculum or delivery and how CDP can improve program performance and document how the training was applied, and how it impacts individuals, organizations and mission operations. FEMA Forms 092-0-2(A) and 092-0-2(B) (explained in paragraph “c” below, were revised to incorporate five common questions used by other NTED components. Also, to reduce burden and eliminate repeat questions on other surveys all questions regarding CDP facilities, transportation and other mission support activities have been removed.

3. Post Graduate Evaluation Form for Supervisors, FEMA Form 092-0-2(B): This form is used to capture the degree which course material affects the performance of students in their place of employment as viewed by their supervisors. The data is used to assess the validity and relativity of the course material and student responses as it relates to actual performance of duties. The supervisor working with and observing the employee can objectively determine if and/or how the training has improved the employee’s performance and how this training can impact and improve their organization through changes to plans, policies, procedures, operations and/or other protocols.

4. EMI: FEMA Form 519-0-1 (5/15) “FEMA Follow-up Evaluation Survey” Similarly, to the entries above EMIs Level 3 Student Follow-up Evaluation Survey Form – FEMA Form 519-0-1 is used to collect quantitative and qualitative information to determine whether students are applying knowledge and skills obtained in training in their work places. To measure the level of retention and or application of new knowledge and skills gain through training approx. 3 – 6 months following training the level 3 is distributed by EMI via email to graduates of their National Emergency Training Center (NETC) resident and mobile training courses. The EMI does this by releasing an email to each class/course cohort with instructions on how to complete the seven question .PDF file survey. The survey contains voluntary responses to both quantitative and qualitative questions and incorporates five common questions used by other NTED components to allow for data aggregation and analysis across the National Preparedness Training portfolio. Once graduates complete their survey, they return it to the EMI via email for data aggregation and analysis by Scantron© software. EMI, CDP and TPP all share the training data and information they obtain from their Level 3s with respective stakeholder groups during three-year course/program reviews. Other stakeholders who receive this data information are NPD, FEMA Executive Leadership, OMB, and Congress.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

1. TPP response: This is a new data collection for TPP and their NDPC and CTG partners. Collectively the TPP and its partners participated in the design of a new Level 3 Post Course Assessment Form that will be delivered by each partner to their respective course/program graduates by class cohort. Technology involves distribution and collection of training data and information via email with data aggregation using Micro-Strategy© software licensed by the NPD-TPP Branch.

Beginning 1st Qtr, FY 2020 quarterly level 3 performance from all three NTED components will be shared with the NTED Analytics Team so that it may be included in the NTED performance metrics – this data sharing is made possible using the FEMA NTED Shared Drive.

2. CDP: Level 3 Post Graduate Evaluation Form for Students is an electronic survey instrument sent to the student approximately 6 - 9 months following the student’s completion of a CDP course. A reminder email is sent to graduates approximately 1 - 2 weeks before the evaluation is due in the event the survey instrument has not already been submitted. The email message describes the importance of completion of the evaluation instrument and provides information for accessing the instrument using the student’s FEMA SID (Student Identification) account number. The electronic format for this evaluation was determined to be the least burdensome, as users can access, complete, and submit the instruments at their convenience with minimal effort. The Questionmark© software is used for electronic delivery of the Level 3 evaluations. This software was purchased to reduce the burden and create ease of completion for students.

CDP: Level 3 Post Graduate Evaluation Form for Supervisors is an electronic survey instrument sent to the student’s supervisor approximately 6 - 9 months following the student’s completion of a CDP course. A reminder email is sent to the supervisor approximately 1 - 2 weeks before the evaluation is due in the event the survey instrument has not already been submitted. The email message describes the importance of completing the evaluation instrument and provides information on completing and submitting the form. The form is sent directly to the supervisor using their email or physical address. Electronic format for this evaluation was determined to be the least burdensome, as users can access, complete, and submit the instruments at their convenience with minimal effort. Questionmark© software is used for electronic delivery of the Level 3 evaluations.

3. EMI: FEMA Follow-up Evaluation Survey. Prior to submitting FEMA Form 519-01-01 in 2017 to the FEMA Forms Office, the EMI delivered paper Level 3 forms to graduates of NETC resident and non-resident mobile training teams via U.S. Postal Service courier. They have since, beginning 2017, instituted a hybrid system leveraging email and technology to reduce cost and burden. As explained previously, in lieu of paper distribution via U.S. Postal Service, EMI now emails their Level 3 survey with instructions to graduates 3 – 6 months following training. Instructions include how to complete, save, and return individual .PDF survey responses to NETC for centralized data and information aggregation and analysis. Data aggregation is performed via Scantron© software and office scanners.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above. 

This information is not collected by other sources or forms, and therefore is not duplicated elsewhere. In 2016 the NTED Director convened a Working Group of Subject Matter Experts (SMEs) from CDP, EMI and TPP to assess how to effectively collect Level 3 data. A determination was made as long as the three components contain a minimum of four to five common data fields on their level 3s to assess NTED portfolio performance they should retain the flexibility to customize their forms to meet the needs and expectations of their specific programs and unique audiences/occupations.

5. If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize.

This information collection does not have an impact on small businesses or other small entities.

6. Describe the consequence to Federal/FEMA program or policy activities if the collection of information is not conducted or is conducted less frequently as well as any technical or legal obstacles to reducing burden.

These course evaluation instruments are meant to provide systematic feedback for TPP, CDP, and EMI courses, enabling program managers to potentially improve course material, delivery, overall program effectiveness, and to inform FEMA’s executive team and Congress on national preparedness capabilities. Ultimately, the information and data provides NTED, NPD, FEMA, OMB and Congressional Leadership empirical data or indicators on the Nation’s overall preparedness level and ensures these programs are compliant with the 2010 GPRA Modernization Act of 2010 § 1115 (A) Fed. Gov. and (B) Agency Performance Plans (6). Without this data, it is difficult to determine operational and strategic needs for improvements for each course/program delivery to meet the nation’s disaster capability/response needs.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

1. Requiring respondents to report information to the agency more often than quarterly. N/A

 (b) Requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it. N/A

3. Requiring respondents to submit more than an original and two

copies of any document. N/A

3. Requiring respondents to retain records, other than health,

medical, government contract, grant-in-aid, or tax records for more than three years. N/A

3. In connection with a statistical survey, that is not designed to

produce valid and reliable results that can be generalized to the universe of study. N/A

 (f) Requiring the use of a statistical data classification that has not

been reviewed and approved by OMB. N/A

 (g) That includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use. N/A

 (h) Requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law. N/A

8. Federal Register Notice: 

 a. Provide a copy and identify the date and page number of publication in the Federal Register of the agency’s notice soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

A 60-day Federal Register Notice inviting public comments was published on May 26, 2020, 85 FR 31538. Two non-germane comments were received.

A 30-day Federal Register Notice inviting public comments was published on November 25, 2020 85 FR 75349.

A 30-day Federal Register Notice correctioninviting public comments was published on December 18, 2020 85 FR 82496.

 b. Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

Information Technology, the CDP’s contractor for Training Delivery, and the CDP’s contractor for Student Services were consulted regarding instrument format, content, delivery and submission. Government staff were also consulted as to the type of data they are needing in order the make improvements in their work toward meeting the agency’s mission.

CDP, EMI and TPP are component organizations of the National Training and Education Division (NTED). All three organizations consult with each other, their parent organization, and their respective stakeholders on the utilization of these evaluation forms to obtain their views on the availability of data, frequency of collection and clarity of instructions. The results of the data collected are shared with their respective audiences during scheduled three year course reviews, and/or audits. All NTED Training organizations (CDP, EMI and TPP) use the DHS and OMB endorsed Four Level Kirkpatrick Model of Training Program Evaluation which requires four levels, or continuous engagement with stakeholders before, during and after training to access end-user, supervisor and organizational reaction, level of learning, change in behavior, and return on expectations.

c. Describe consultations with representatives of those from whom information is to be obtained or those who must compile records. Consultation should occur at least once every three years, even if the collection of information activities is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

Consultation with those from whom information is to be obtained is continuous, as representatives from the Training Support and Student Services department talk directly to first responders and receivers in “Meet and Greet” sessions held every Tuesday or Wednesday evening in the lodging area at CDP. Reports of these meetings are developed and distributed to various departments for possible new course development, technology changes and other recommendations toward improving CDP’s overall services to the student’s needs. Comments regarding the forms and instructions are directed to CDP’s Curriculum and Evaluation department for future modifications of the instruments.

Similar to the CDP response above, both EMI and TPP also consult with their respective audience groups or customers on a periodic and three-year cyclic basis. Instructional Systems Design ISD, as well as the Kirkpatrick Model are systems approaches to provide continuous consultation with customers (students, their supervisors and decision makers) while training is designed, developed, piloted and delivered. The four Training Evaluation forms included in this submission provide data-based decisions obtained from continuous customer, product and process feedback for the FEMA NTED National Preparedness Training mission.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

NTED and its component organizations (CDP, EMI and TPP) do not provide payments or gifts to respondents in exchange for a benefit sought.

10. Describe any assurance of confidentiality provided to respondents. Present the basis for the assurance in statute, regulation, or agency policy.

A Privacy Threshold Analysis (PTA) was approved by the DHS Privacy Office on August 31, 2020.

11. Provide additional justification for any question of a sensitive nature (such as sexual behavior and attitudes, religious beliefs and other matters that are commonly considered private). This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

There are no questions of a sensitive nature required in this data collection.

 12. Provide estimates of the hour burden of the collection of information. The statement should:

 a. Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated for each collection instrument (separately list each instrument and describe information as requested). Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desired. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.

 b. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.

c. Provide an estimate of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. NOTE: The wage-rate category for each respondent must be multiplied by 1.4 and this total should be entered in the cell for “Avg. Hourly Wage Rate”. The cost to the respondents of contracting out or paying outside parties for information collection activities should not be included here. Instead this cost should be included in Item 13.

Estimated Annualized Burden Hours and Costs
Type of Respondent	Form Name / Form No.	No. of Respondents	No. of Responses per Respondent	Total No. of Responses	Avg. Burden per Response (in hours)	Total Annual Burden (in Hours)	Avg. Hourly Wage Rate	Total Annual Respondent Cost
State, local, or Tribal government (Protective Services)	TPP Level 3 Post Course Assessment FEMA Form 016-0-2	107,948	1	107,948	0.25	26,987	$46.29	$1,249,228
State government (Emergency Management Directors)	TPP Level 3 Post Course Assessment FEMA Form 016-0-2	4,266	1	4,266	0.25	1,067	$54.10	$57,725
Private Sector (Protective Services)	TPP Level 3 Post Course Assessment, FEMA Form 016-0-2	4	1	4	0.25	1	$24.09	$24
State, local, or Tribal government (Protective Services)	CDP Level 3 Post Graduate Evaluation for Students, FEMA Form 092-0-2A	4,000	1	4,000	0.25	1,000	$45.10	$45,100
State, local, or Tribal government (Healthcare Practitioners and Technical occupation)	CDP Level 3 Post Graduate Evaluation for Students, FEMA Form 092-0-2A	2,000	1	2,000	0.25	500	$57.62	$28,810
State, local, or Tribal government (Protective Services)	CDP Level 3 Post Graduate Evaluation for Supvs, FEMA Form 092-0-2B	400	1	400	0.25	100	$45.10	$4,510
State, local, or Tribal government (Healthcare Providers and Technical Occupations)	CDP Level 3 Post Graduate Evaluation for Supvs, FEMA Form 092-0-2B	200	1	200	0.25	50	$57.62	$2,881
State government (Emergency Management Directors)	EMI Post Course Assessment form, FEMA Form 519-0-1	5,874	1	5,874	0.25	1,469	$54.10	$79,473
Total		124,692		124,692		31,174		$1,467,751

Instruction for Wage-rate category multiplier: Take each non-loaded “Avg. Hourly Wage Rate” from the BLS website table and multiply that number by 1.6¹. For Private Sector and Federal workers, multiply the wage rate by 1.46² For example, a non-loaded BLS table wage rate of $42.51 would be multiplied by 1.6, and the entry for the “Avg. Hourly Wage Rate” would be $68.02.

TPP Level 3 Post Course Assessment Form 016-0-2:

According to the U.S. Department of Labor, Bureau of Labor Statistics³, the May 2019 Occupational Employment and Wage Estimates wage rate for Protective Services working for State governments (SOC 33-0000) is $28.93. Including the wage rate multiplier of 1.6, the fully-loaded wage rate is $46.29 per hour. Therefore, the annual burden hour cost is estimated to be $1,249,228 ($46.29 x 26,987 hours).

According to the U.S. Department of Labor, Bureau of Labor Statistics, the May 2019 Occupational Employment and Wage Estimates wage rate for Emergency Management Directors working for State governments (SOC 11-9161) is $33.81. Including the wage rate multiplier of 1.6, the fully-loaded wage rate is $54.10 per hour. Therefore, the annual burden hour cost is estimated to be $57,724.750 ($54.1053.70 x 1,067 hours).

According to the U.S. Department of Labor, Bureau of Labor Statistics, the May 2019 Occupational Employment and Wage Estimates wage rate for Protective Services in the private sector (SOC 33-0000) is $16.50. Including the wage rate multiplier of 1.46, the fully-loaded wage rate is $24.09 per hour. Therefore, the annual burden hour cost is estimated to be $24.09 ($24.09 x 1 hour).

CDP Level 3 Post Course Graduate Evaluation for Students Form 092-0-2A:

According to the U.S. Department of Labor, Bureau of Labor Statistics, the May 2019 Occupational Employment and Wage Estimates wage rate for Protective Services State, local, or Tribal government (SOC 33-0000) is $28.19. Including the wage rate multiplier of 1.6, the fully-loaded wage rate is $45.10 per hour. Therefore, the annual burden hour cost is estimated to be $45,100.00 ($45.10 x 1,000 hours).

According to the U.S. Department of Labor, Bureau of Labor Statistics, the May 2019 Occupational Employment and Wage Estimates wage rate for Healthcare Practitioners and Technical occupations in State, local, or Tribal governments (SOC 29-0000) is $36.01. Including the wage rate multiplier of 1.6, the fully-loaded wage rate is $57.62per hour. Therefore, the annual burden hour cost is estimated to be $28,810.00 ($57.62 x 500 hours).

CDP Level 3 Post Course Graduate Evaluation for Supervisor Form 092-0-2B:

According to the U.S. Department of Labor, Bureau of Labor Statistics, the May 2019 Occupational Employment and Wage Estimates wage rate for Protective Services State, local, or Tribal governments (SOC 33-0000) is $28.19. Including the wage rate multiplier of 1.6, the fully-loaded wage rate is $45.10 per hour. Therefore, the annual burden hour cost is estimated to be $4,510.00 ($45.10 x 100 hour).

According to the U.S. Department of Labor, Bureau of Labor Statistics, the May 2019 Occupational Employment and Wage Estimates wage rate for Healthcare Providers and Technical occupations in State, local, or Tribal governments (SOC 29-0000) is $36.01. Including the wage rate multiplier of 1.6, the fully-loaded wage rate is $57.62 per hour. Therefore, the annual burden hour cost is estimated to be $2,881.00 ($57.62 x 50 hours).

EMI Level 3 Post Course Assessment Form:

EMI Post grad students form, FEMA Form 519-0-1: According to the U.S. Department of Labor, Bureau of Labor Statistics, the May 2019 Occupational Employment and Wage Estimates wage rate for Emergency Management Directors working for State governments (SOC 11-9161) is $33.81. Including the wage rate multiplier of 1.6, the fully-loaded wage rate is $54.10 per hour. Therefore, the annual burden hour cost is estimated to be $79,472.90 ($53.70 x 1,469 hours).

13. Provide an estimate of the total annual cost burden to respondents or recordkeepers resulting from the collection of information. The cost of purchasing or contracting out information collection services should be a part of this cost burden estimate. (Do not include the cost of any hour burden shown in Items 12 and 14.)

The cost estimates should be split into two components:

a. Operation and Maintenance and purchase of services component. These estimates should take into account cost associated with generating, maintaining, and disclosing or providing information. Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred.

b. Capital and Start-up-Cost should include, among other items, preparations for collecting information such as purchasing computers and software, monitoring sampling, drilling and testing equipment, and record storage facilities.

Annual Cost Burden to Respondents or Recordkeepers
Data Collection Activity/Instrument	*Annual Capital Start-Up Cost (investments in overhead, equipment, and other one-time expenditures	*Annual Operations and Maintenance Costs (such as recordkeeping, technical/professional services, etc.)	Annual Non-Labor Cost (expenditures on training, travel, and other resources)	Total Annual Cost to Respondents
Total	$0	$0	$0	$0

There is no anticipated respondent O&M, Capital or Start-up cost associated with this collection. The target audience maintains normal email service which is subject to associated daily organizational O&M costs.

14. Provide estimates of annualized cost to the federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing and support staff), and any other expense that would have been incurred without this collection of information. You may also aggregate cost estimates for Items 12, 13, and 14 in a single table.

Annual Cost to the Federal Government
Item	Cost ($)
Contract Costs [Describe]	$0
Staff Salaries1 1 of GS 9 Step 5 employee spending approximately 90% of time annually to process TPP for the data collection, and 1 GS 11 Step 5 spending approximately 30 % of time annually to review, coordinate, and assure quality control (1 x $63,170 x 0.9 x 1.462 + 1 x $76,428 x .3 x 1.46 = $116,481) At EMI, Direct labor for the full time General Clerk is $35,200	$151,681
Facilities [cost for renting, overhead, etc. for data collection activity]	$0
Computer Hardware and Software [cost of equipment annual lifecycle] Computer hardware and software (Micro-Strategy 55 user licenses + 3 developer licenses) = $15,868.01; for CDP no charge; For EMI $240.00. Total for 3 sites: $15,868.01 + $240.00 = $16,108.01	$16,108
Equipment Maintenance [cost of annual maintenance/service agreements for equipment] Equipment Maintenance (three instances of Level 3 support from March-Aug 2018) is $342.00; no Equipment maintenance for CDP; For EMI, Cost of annual maintenance agreement for the Scantron Op4ES is $782.00. Total is $342.00 + $782.00 = $1,124	$1,124
Travel	$0
Total	$169,913
1 Office of Personnel Management 2020 Pay and Leave Tables for the Atlanta-Athens-Clark County-Sandy Springs, GA-AL Locality. Available online at https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/20Tables/html/ATL.aspx. Accessed April 20, 2020. 2 Wage rate includes a 1.46 multiplier to reflect the fully-loaded wage rate.

15. Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-I in a narrative form. Present the itemized changes in hour burden and cost burden according to program changes or adjustments in Table 5. Denote a program increase as a positive number, and a program decrease as a negative number.

A "Program increase" is an additional burden resulting from a federal government regulatory action or directive. (e.g., an increase in sample size or coverage, amount of information, reporting frequency, or expanded use of an existing form). This also includes previously in-use and unapproved information collections discovered during the ICB process, or during the fiscal year, which will be in use during the next fiscal year.

A "Program decrease", is a reduction in burden because of: (1) the discontinuation of an information collection; or (2) a change in an existing information collection by a Federal agency (e.g., the use of sampling (or smaller samples), a decrease in the amount of information requested (fewer questions), or a decrease in reporting frequency).

"Adjustment" denotes a change in burden hours due to factors over which the government has no control, such as population growth, or in factors which do not affect what information the government collects or changes in the methods used to estimate burden or correction of errors in burden estimates.

Itemized Changes in Annual Burden Hours
Data Collection Activity/Instrument	Program Change (hours currently on OMB inventory)	Program Change (new)	Difference	Adjustment (hours currently on OMB inventory)	Adjustment (new)	Difference
Post Course Assessment Form NTED-TPP	0	28,988	28,988			0
Total	0	28,988	28,988	0	0	0

Explain: TPP “Post Course Assessment Form NTED-TPP: The TPP Level 3 Post Course Assessment Form is new and, therefore, increases burden hours; other burden hours for EMI and CDP remain the same.

Itemized Changes in Annual Cost Burden
Data Collection Activity/Instrument	Program Change (cost currently on OMB inventory)	Program Change (new)	Difference	Adjustment (cost currently on OMB inventory)	Adjustment (new)	Difference
TPP collections/new forms	$0	$1,190,953	$1,190,953			$0
EMI lowered postal costs	$6,461	$0	-$6,461			$0
Total	$6,461	$1,190,953	$1,197,414	$0	$0	$0

Explain: For TPP, the new Level 3 Post Course Assessment Form incurs annual burden costs of $1,190,953.12. This is a total of all the TPP additions to the collection ($252,859.07+$830,983.10+$11,840.91+$33.12+$51,117.35+$44,119.57=$1,190,953.12). For EMI there is no change in respondent burden hours or costs however, because EMI has adopted use of email verses continued use of U.S. Postal Service to deliver and collect its new Level 3 form to course graduates there is a cost savings or reduction in annual cost for U.S. Gov.:

The 2020 U.S. Postal Service fee to deliver 5,874 Level 3 surveys to EMI course graduates was .55 per survey = $3,230.70 X 2 for .55 cent return postal fee = -$6,461.40 annual cost savings by using email distribution.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

There are no outline plans for tabulation and publication of data for this information collection.

17. If seeking approval not to display the expiration date for OMB approval of the information collection, explain reasons that display would be inappropriate.

This collection does not seek approval to not display the expiration date for OMB approval.

18. Explain each exception to the certification statement identified in Item 19 “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-I.

Adjust this sentence as necessary: This collection does not seek exception to “Certification for Paperwork Reduction Act Submissions”. Statistical Survey methodology "is not applicable".

Appendix A:

1 Bureau of Labor Statistics, Employer Costs for Employee Compensation, Table 1.  “Employer costs per hour worked for employee compensation and costs as a percent of total compensation:  Civilian workers, by major occupational and industry group, March 2019.”  Available at http://www.bls.gov/news.release/archives/ecec_06182019.pdf.  Accessed April 21, 2020.  The wage multiplier is calculated by dividing total compensation for State and local government workers of $50.89 by Wages and salaries for State and local government workers of $31.75 per hour yielding a benefits multiplier of approximately 1.6

2 Bureau of Labor Statistics, Employer Costs for Employee Compensation, Table 1.  “Employer costs per hour worked for employee compensation and costs as a percent of total compensation:  Civilian workers, by major occupational and industry group, March 2019.”  Available at http://www.bls.gov/news.release/archives/ecec_06182019.pdf.  Accessed April 21, 2020.  The wage multiplier is calculated by dividing total compensation for all workers of $36.77 by wages and salaries for all workers of $25.22 per hour yielding a benefits multiplier of approximately 1.46

3 Information on the mean wage rate from the U.S. Department of Labor Bureau of Labor Statistics is available online at: https://www.bls.gov/oes/tables.htm

9