10/24/2014
Supporting Statement for
Paperwork Reduction Act Submissions
OMB Control Number: 1660-0132
Title: Level 1 End-of Course Evaluation and Level 3 Post Graduate Evaluations for the Center for Domestic Preparedness (CDP)
Form Number(s):
Level 1 End-of-Course Evaluation Form, FEMA Form 092-0-2
Level 3 Post Graduate Evaluation Form for Students, FEMA Form 092-0-2A
Level 3 Post Graduate Evaluation Form for Supervisors, FEMA Form 092-0-2B
A Supporting Statement, including the text of the notice to the public required by 5 CFR 1320.5(a)(i)(iv) and its actual or estimated date of publication in the Federal Register, must accompany each request for approval of a collection of information. The Supporting Statement must be prepared in the format described below, and must contain the information specified in Section A below. If an item is not applicable, provide a brief explanation. When Item 17 or the OMB Form 83-I is checked “Yes”, Section B of the Supporting Statement must be completed. OMB reserves the right to require the submission of additional information with respect to any request for approval.
Explain the circumstances that make the collection of information necessary.
Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information. Provide a detailed description of the nature and source of the information to be collected.
Section 1204 of the Implementing Recommendations of the 9/11 Commission Act of 2007, Public Law 110-53, 121 Stat. 266, (codified at 6 U.S.C. § 1102), authorizes the Secretary of Homeland Security to establish, operate, and maintain a National Domestic Preparedness Consortium (NDPC) within the Department of Homeland Security. Section 1204 specifically recognizes the Federal Emergency Management Agency (FEMA’s) Center for Domestic Preparedness (CDP) in Anniston, Alabama as a component of the NDPC, with a statutory mandate to “identify, develop, test, and deliver training to State, Local, Tribal, and Territorial emergency response providers, provide on-site and mobile training at the performance and management and planning levels, and facilitate the delivery of training by the training partners of the Department.” In accordance with those Congressional mandates, the CDP offers courses and programs that are delivered on-campus at the CDP facility and throughout the Nation in coordination with State, local, and Tribal emergency response jurisdictions, as well as other training partners of the Department of Homeland Security. The collection of applicant information necessary to register for courses and to receive reimbursement for travel for those courses is necessary to meet the Congressional mandates. The level 1 data collected during the training will provide invaluable information about the student’s perception or reaction to the training with suggestions for improvements and changes to the curriculum and training. The level 3 data provides CDP with data about how the training was transferred to the student’s job and organization and if the training was applied to a particular event whether an exercise or response to a real event.
2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection. Provide a detailed description of: how the information will be shared, if applicable, and for what programmatic purpose.
The CDP will utilize the Kirkpatrick Four Levels of Evaluation. In order to assess training effectiveness, Donald Kirkpatrick developed a four level model of evaluation. Level 1 evaluation captures the students perception of training, level 2 is the cognitive Evaluation (tests/examinations and performance assessment checklists) of the student learning (change in behavior), level 3 evaluation instrument captures how the training transferred to the student’s work environment and their particular job, and level 4 captures information about the return on investment of the training program. This supporting statement includes the collection of level 1 and level 3 evaluation data.
The level 1 instrument captures student’s reaction to the training. This level produces data regarding the relevancy of the training for the student and also captures their confidence level regarding their abilities to perform the skills acquired in the training.
Level 1 End-of-Course Evaluation Form, FEMA Form 092-0-2: The Level 1 instrument is used to survey the Center for Domestic Preparedness (CDP) students’ enrolled in CDP courses. The survey collects information regarding the increase of knowledge and skills the student acquires after taking a CDP course. It also collects information regarding the quality of instruction and the increase in confidence the student has acquired from the training. There are also questions regarding the training facility and student services. The CDP analyzes the data to determine what revisions are required in the course materials, the instruction, and other services.
Level 3 evaluations collect data on whether the training has been transferred to the student’s work. This Post Graduate Evaluation provides a reflection as to the training program’s effectiveness and how training impacted the student and their organization in making improvements as a result of the training.
b. Level 3 Post Graduate Evaluation Form for Students, FEMA Form 092-0-2A: This form is used to capture the degree to which the course material affected the performance of the student in their professional employment. The data is used to assess the validity and relativity of the course material as it relates to the actual performance of duties. The CDP analyzes the data to determine what revisions are needed in curriculum, improvements to instruction, how CDP can improve overall services, and provide documentation as to how the training was applied and impacted the individual and organization.
c. Level 3 Post Graduate Evaluation Form for Supervisors, FEMA Form 092-0-2B: This form is used to capture the degree to which the course material affected the performance of the student in their professional employment. The data is used to assess the validity and relativity of the course material as it relates to the actual performance of duties. The Supervisor working with and observing the employee can ascertain how the training has improved the employee’s performance and how this training can impact and improve their organization through changes to plans, policies, procedures, and protocols.
3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.
a. Level 1 End-of Course Evaluation Form —The Level 1 instrument will be paper collection for non-resident courses and use an electronic collection system for resident courses. The use of any form of information technology is not feasible for this Evaluation for offsite (nonresident) course deliveries as computer access may not be readily available
b. Level 3 Post Graduate Evaluation Form for Students — This form is an electronic survey instrument sent to the student approximately six to nine months following the student’s completion of a CDP course. A reminder email is sent to the student approximately one to two weeks before the evaluation is due in the event the survey instrument has not already been submitted. The email message describes the importance of completion of the Evaluation instrument and provides information for accessing the instrument using the student’s FEMA SID (Student Identification) account number. The electronic format for this Evaluation was determined to be the least burdensome, as users can access, complete and submit the instruments at their convenience with minimal effort.
c. Level 3 Post Graduate Evaluation Form for Supervisors — This form is an electronic survey instrument sent to the student’s Supervisor approximately six to nine months following the student’s completion of a CDP course. A reminder email is sent to the supervisor approximately one to two weeks before the evaluation is due in the event the survey instrument has not already been submitted. The email message describes the importance of completion of the Evaluation instrument and will provide information on completing and submitting the form. The form is sent directly to the supervisor using their email or physical address. Electronic format for this Evaluation was determined to be the least burdensome, as users can access, complete and submit the instruments at their convenience with minimal effort.
4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.
This information is not collected in any form, and therefore is not duplicated elsewhere.
5. If the collection of information impacts small businesses or other small entities (Item 5 of OMB Form 83-I), describe any methods used to minimize.
This information collection does not have an impact on small businesses or other small entities.
6. Describe the consequence to Federal/FEMA program or policy activities if the collection of information is not conducted, or is conducted less frequently as well as any technical or legal obstacles to reducing burden.
These course evaluation instruments are meant to give feedback to CDP that can potentially improve the course material, delivery, overall program effectiveness, and also provide empirical data about the Nation’s overall sense and confidence of preparedness. Without this data, it is difficult to determine the need for improvements, degree of student satisfaction, and instructor capability for each course delivery.
7. Explain any special circumstances that would cause an information collection to be conducted in a manner:
Requiring respondents to report information to the agency more
often than quarterly.
(b) Requiring respondents to prepare a written response to a
collection of information in fewer than 30 days after receipt of it.
Requiring respondents to submit more than an original and two
copies of any document.
Requiring respondents to retain records, other than health,
medical, government contract, grant-in-aid, or tax records for more than three years.
In connection with a statistical survey, that is not designed to
produce valid and reliable results that can be generalized to the universe of study.
(f) Requiring the use of a statistical data classification that has not
been reviewed and approved by OMB.
(g) That includes a pledge of confidentiality that is not supported by
authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use.
(h) Requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.
The special circumstances contained in item 7 of the supporting statement are not applicable to this information collection.
8. Federal Register Notice:
a. Provide a copy and identify the date and page number of publication in the Federal Register of the agency’s notice soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.
A 60-day Federal Register Notice inviting public comments was published on November 19, 2014, 79 FR 68896. No comments were received. See attached copy of the published notice included in this package.
A 30-day Federal Register Notice inviting public comments was published on March 19, 2015, 80 FR 14404. No comments were received. See attached copy of the published notice included in this package.
b. Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.
In preparation of the forms and instructions, CDP’s contractor for Information Technology, the CDP’s contractor for training delivery, and the CDP’s contractor for Student Services were consulted regarding instrument format, content, delivery and submission. Government staff were also consulted as to the type of data they are needing in order the make improvements in their work toward meeting the agency’s mission.
c. Describe consultations with representatives of those from whom information is to be obtained or those who must compile records. Consultation should occur at least once every three years, even if the collection of information activities is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.
Consultation with those from whom information is to be obtained is continuous, as representatives from the Training Support and Student Services department talk directly to first responders and receivers in “Meet and Greet” sessions held every Tuesday or Wednesday evening in the Lodging area. Reports of these meetings are developed and distributed to various departments for possible new course development, technology changes and other recommendations toward improving CDP’s overall services to the student’s needs. Comments regarding the forms and instructions are directed to CDP’s Curriculum and Evaluation department for future modifications of the instruments.
9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.
FEMA does not provide payments or gifts to respondents in exchange for a benefit sought.
10. Describe any assurance of confidentiality provided to respondents. Present the basis for the assurance in statute, regulation, or agency policy.
A Privacy Threshold Analysis (PTA) was submitted to the FEMA Privacy office on 8-23-2014 in order to determine if a Privacy Impact Analysis (PIA) for this collection is needed to also be forwarded to the FEMA Privacy Office for review. The PTA status is under review as of August 2014.
11. Provide additional justification for any question of a sensitive nature (such as sexual behavior and attitudes, religious beliefs and other matters that are commonly considered private). This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.
There are no questions of a sensitive nature required in this data collection.
12. Provide estimates of the hour burden of the collection of information. The statement should:
a. Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated for each collection instrument (separately list each instrument and describe information as requested). Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desired. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.
It is anticipated that 25,500 State, Local, or Tribal, government respondents will complete the Level 1 End-of-course Evaluation Form. Each respondent will only complete the form once and each response will require 0.25 (15 minutes) hours to complete the form. The total annual hour burden is 25,500 x 0.25 hours = 6,375 annual hours.
It is anticipated that 12,500 healthcare professional and technical respondents for State, Local, or Tribal, governments will complete the Level 1 End-of-course Evaluation Form. Each respondent will only complete the form once and each response will require 0.25 (15 minutes) hours to complete the form. The total annual hour burden is 12.500 x 0.25 hours = 3,125 annual hours
It is anticipated that 4,000 first responder respondents for State, Local, or Tribal, governments will complete the Level 3 Post Graduate Evaluation Form for Students. Each respondent will only complete the form once and each response will require 0.25 hours (15 minutes to complete the form). The total annual hour burden is 4000 x .25 hours = 1000 annual hours.
It is anticipated that 2,000 healthcare professional and technical respondents for State, Local, or Tribal governments will complete Level 3 post Graduate Evaluation Form for Students. Each respondent will only complete the form once and each response will require 0.25 (15 minutes) hours to complete the form. The total annual hour burden is 2000 x 0.25 hours = 500 annual hours.
It is anticipated that 400 first responder respondents for State, Local, or Tribal,governments will complete the Level 3 Post Graduate Evaluation Form for Supervisors. Each respondent will only complete the form once and each response will require 0.25 hours (15 minutes) to complete the form. The total annual hour burden is 400 x 0.25 hours = 100 annual hours.
It is anticipated that 200 healthcare professional and technical respondents for State, Local, or Tribal, governments will complete the Level 3 Post Graduate Evaluation Form for Supervisors. Each respondent will only complete the form once and each response will require 0.25 hours (15 minutes) to complete the form. The total annual hour burden is 200 x 0.25 hours = 50 annual hours.
b. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.
c. Provide an estimate of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost to the respondents of contracting out or paying outside parties for information collection activities should not be included here. Instead this cost should be included in Item 13.
Table A.12: Estimated Annualized Burden Hours and Costs |
||||||||
Type of Respondent |
Form Name / Form Number |
No. of Respon-dents |
No. of Respon-ses per Respon-dent |
Total No. of Responses |
Avg. Burden per Response (in hours) |
Total Annual Burden (in hours) |
Avg. Hourly Wage Rate* |
Total Annual Respondent Cost |
State, Local, Tribal, or Territorial government |
Level 1 End-of-course Evaluation, FEMA Form 092-0-2 |
25,500 |
1 |
25,500 |
15 minutes (.25 hours) |
6,375 |
$29.29 |
$186,723.75 |
Healthcare Professionals and Technical |
Level 1 End-of course Evaluation, FEMA Form 092-0-2 |
12,500 |
1 |
12,500 |
15 minutes (.25 hours) |
3,125 |
$50.30 |
$157,187.50 |
State, local or Tribal government |
Level 3 Post Graduate Evaluation for Students, FEMA Form 092-0-2A |
4,000 |
1 |
4,000 |
15 minutes (.25 hours) |
1.000 |
$29.29 |
$29,290.00 |
Healthcare Professionals and Technical |
Level 3 Post Graduate Evaluation for Students, FEMA Form 092-0-2A |
2,000 |
1 |
2,000 |
15 minutes (.25 hours) |
500 |
$50.30 |
$25,150.00 |
State, local or Tribal government |
Level 3 Post Graduate Evaluation for Supvs, FEMA Form 092-0-2B |
400 |
1 |
400 |
15 minutes (.25 hours |
100 |
$29.29 |
$2,929.00 |
Healthcare Professionals and Technical |
Level 3 Post Graduate Evaluation for Supvs, FEMA Form 092-0-2B |
200 |
1 |
200 |
15 minutes (.25 hours |
50 |
$50.30 |
$2,515.00 |
Total |
|
44,600 |
|
|
|
11,150 |
|
$403,795.25 |
Note: The “Avg. Hourly Wage Rate” for each respondent includes a 1.4 multiplier to reflect a fully-loaded wage rate.
“Type of Respondent” should be entered exactly as chosen in Question 3 of the OMB Form 83-I
Instruction for Wage-rate category multiplier: Take each non-loaded “Avg. Hourly Wage Rate” from the BLS website table and multiply that number by 1.4. For example, a non-loaded BLS table wage rate of $42.51 would be multiplied by 1.4, and the entry for the “Avg. Hourly Wage Rate” would be $59.51.
FEMA Form 092-0-2:
According to the U.S. Department of Labor, Bureau of Labor Statistics website (www.bls.gov) the wage rate category for Protective Services for Federal State and Local is estimated to be $20.92 per hour and multiplied by 1.4, the “Avg. Hourly Wage Rate” would be $29.29, therefore, the estimated burden hour cost to respondents from the Protective Services is estimated to be 6,375 hours x $29.29 = $186,723.75 annually.
FEMA Form 092-0-2:
According to the U.S. Department of Labor, Bureau of Labor Statistics website (www.bls.gov) the mean wage rate category for Healthcare Providers and Technical occupations in Federal, State and Local industry is estimated to be $35.93 per hour and multiplied by 1.4, the “Avg. Hourly Wage Rate” would be $50.30, therefore, the estimated burden hour cost to respondents in this category is estimated to be 3,125 hours x $50.30 = $157,187.50 annually.
FEMA Form 092-0-2A:
According to the U.S. Department of Labor, Bureau of Labor Statistics website (www.bls.gov) the wage rate category for Protective Services for Federal State and Local is estimated to be $20.92 per hour and multiplied by 1.4, the “Avg. Hourly Wage Rate” would be $29.29, therefore, the estimated burden hour cost to respondents from the Protective Services is estimated to be 1000 hours x $29.29= $29,290 annually.
FEMA Form 092-0-2A:
According to the U.S. Department of Labor, Bureau of Labor Statistics website (www.bls.gov) the mean wage rate category for Healthcare Providers and Technical occupations in Federal, State and Local industry is estimated to be $35.93 per hour and multiplied by 1.4, the “Avg. Hourly Wage Rate” would be $50.30, therefore, the estimated burden hour cost to respondents in this category is estimated to be 500 hours x $50.30= $25,150 annually.
FEMA Form 092-0-2B:
According to the U.S. Department of Labor, Bureau of Labor Statistics website (www.bls.gov) the wage rate category for Protective Services for Federal State and Local is estimated to be $20.92 per hour and multiplied by 1.4, the “Avg. Hourly Wage Rate” would be $29.29, therefore, the estimated burden hour cost to respondents from the Protective Services is estimated to be 100 hours x $29.29= $2,929 annually.
FEMA Form 092-0-2B:
According to the U.S. Department of Labor, Bureau of Labor Statistics website (www.bls.gov) the mean wage rate category for Healthcare Providers and Technical occupations in Federal, State and Local industry is estimated to be $35.92 per hour and multiplied by 1.4, the “Avg. Hourly Wage Rate” would be $50.30, therefore, the estimated burden hour cost to respondents in this category is estimated to be 50 hours x $50.30= $2,515 annually.
13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. The cost of purchasing or contracting out information collection services should be a part of this cost burden estimate. (Do not include the cost of any hour burden shown in Items 12 and 14.)
The cost estimates should be split into two components:
a. Operation and Maintenance and purchase of services component. These estimates should take into account cost associated with generating, maintaining, and disclosing or providing information. Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred.
b. Capital and Start-up-Cost should include, among other items, preparations for collecting information such as purchasing computers and software, monitoring sampling, drilling and testing equipment, and record storage facilities.
There is no capital, start-up-cost, or recording keeping costs associated with this information collection.
14. Provide estimates of annualized cost to the federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing and support staff), and any other expense that would have been incurred without this collection of information. You may also aggregate cost estimates for Items 12, 13, and 14 in a single table.
Annual Cost to the Federal Government
* Note: The “Salary Rate” includes a 1.4 multiplier to reflect a fully-loaded wage rate.
|
|
15. Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-I in a narrative form. Present the itemized changes in hour burden and cost burden according to program changes or adjustments in Table 5. Denote a program increase as a positive number, and a program decrease as a negative number.
A "Program increase" is an additional burden resulting from a federal government regulatory action or directive. (e.g., an increase in sample size or coverage, amount of information, reporting frequency, or expanded use of an existing form). This also includes previously in-use and unapproved information collections discovered during the ICB process, or during the fiscal year, which will be in use during the next fiscal year.
A "Program decrease", is a reduction in burden because of: (1) the discontinuation of an information collection; or (2) a change in an existing information collection by a Federal agency (e.g., the use of sampling (or smaller samples), a decrease in the amount of information requested (fewer questions), or a decrease in reporting frequency).
"Adjustment" denotes a change in burden hours due to factors over which the government has no control, such as population growth, or in factors which do not affect what information the government collects or changes in the methods used to estimate burden or correction of errors in burden estimates.
Itemized Changes in Annual Burden Hours |
||||||
Data collection Activity/Instrument |
Program Change (hours currently on OMB Inventory) |
Program Change (New) |
Difference |
Adjustment (hours currently on OMB Inventory) |
Adjustment (New) |
Difference |
Level 1 End-of Course Evaluation , FEMA Form 092-0-2 |
|
|
|
13,000 |
6,375 |
-6,625 |
Level 1 End-of-course Evaluation FEMA Form 092-0-2 |
|
|
|
3,250 |
3,125 |
-125 |
Level 3 Post Graduate Evaluation for Students, FEMA Form 092-0-2A |
|
|
|
375 |
500 |
+125 |
Level 3 Post Graduate Evaluation for Students, FEMA Form 092-0-2A |
|
|
|
500 |
1000 |
+500 |
Level 3 Post Graduate Evaluation for Supervisors, FEMA Form 092-0-2B |
|
|
|
375 |
50 |
-325 |
Level 3 Post Graduate Evaluation for Supervisors, FEMA Form 092-0-2B |
|
|
|
500 |
100 |
-400 |
Total(s) |
|
|
|
18,000 |
11,150 |
-6850 |
Explain: CDP has decreased the population of non-resident training, and increased the population of resident training which affects all the evaluation instruments in this collection.
Itemized Changes in Annual Cost Burden |
||||||
Data collection Activity/Instrument |
Program Change (cost currently on OMB Inventory) |
Program Change (New) |
Difference |
Adjustment (cost currently on OMB Inventory) |
Adjustment (New) |
Difference |
Level 1 End-of-course Evaluation, FEMA Form 092-0-2 |
|
|
|
$446,290.00 |
$186,723.75 |
-$259,566.25 |
Level 1 End-of-Course Evaluation FEMA Form 092-0-2 |
|
|
|
$144,690.00 |
$157,187.50 |
+$12497.50 |
Level 3 Post Graduate Evaluation for Students, FEMA Form 092-0-2A |
|
|
|
$12,873.75 |
$25,150.00 |
+$12,276.25 |
Level 3 Evaluation for Students, FEMA Form 092-0-2A |
|
|
|
$22,260.00 |
$29,290.00 |
+$22,260.00 |
Level 3 Evaluation for Supervisors, FEMA Form 092-0-2B |
|
|
|
$12,873.75 |
$2,515.00 |
-$10,358.75 |
Level 3 Evaluation for Supervisors, FEMA Form 092-0-2B |
|
|
|
$22,260.00 |
$2,929.00 |
-$19,331.00 |
Total(s) |
|
|
|
$661,247.50 |
$ 403,795.25 |
-$242,222.25 |
Explain: The evaluation forms were used for both resident training and indirect training, CDP training completed off-site by non-CDP instructors. CDP is no longer requiring evaluation forms for indirect training. CDP has also decreased the population of non-resident training, and increased the population of resident training which affects all the evaluation instruments in this collection.
16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.
FEMA does not intend to employ the use of statistics or the publication thereof for this information collection.
17. If seeking approval not to display the expiration date for OMB approval of the information collection, explain reasons that display would be inappropriate.
FEMA will display the expiration date for OMB approval of this information collection.
18. Explain each exception to the certification statement identified in Item 19 “Certification for Paperwork Reduction Act Submissions,” of OMB Form 83-I.
FEMA does not request an exception to the certification of this information collection.
Supporting Statement for
Paperwork Reduction Act Submissions
OMB Control Number: 1660 - 0132
Title: Resubmission of Level 1 End-of-Course Evaluation Form, Level 3 Post Graduate Evaluation Form for Students and Level 3 Post Graduate Evaluation Form for Supervisors
Form Number(s):
Level 1 End-of-Course Evaluation Form, FEMA Form 092-0-2
Level 3 Post Graduate Evaluation Form for Students, FEMA Form 092-0-2A
Level 3 Post Graduate Evaluation Form for Supervisors, FEMA Form 092-0-2B
B. Collections of Information Employing Statistical Methods.
When Item 17 on the Form OMB 83-I is checked “Yes”, the following documentation should be included in the Supporting Statement to the extent it applies to the methods proposed:
1. Describe (including numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.
Level 1 End-of-Course Evaluation Form:
This form will be distributed to approximately 38,000 students (annually) at the conclusion of the course completion. The form is a Questionmark® form and will be processed and analyzed with the Questionmark equipment and Questionmark software®. The expected response rate for this collection should be 90-100% as the instrument will be distributed and collected on site. The past response rate was 95%.
Level 3 Post Graduate Evaluation for Students and Level 3 Post Graduate Evaluation for Supervisors:
These forms will be web-based and accessed and submitted through the CDP web-site. CDP has targeted all courses which will total approximately 19,000 students and 9,500 supervisors for this collection. The expected response rate for this collection should be 25% or greater. The past response rate was 31%.
2. Describe the procedures for the collection of information including:
• Statistical methodology for stratification and sample selection,
There is no stratification. All students are asked to take the End-of-course Evaluation after completing their respective courses. The Level 1 survey will be given to all students enrolled in CDP courses at resident and non-resident training sites. It is estimated 38,000 respondents will complete the Level 1. This number was derived through the analysis of past enrollment and projected schedules of courses to be offered in the future.
Level 3 Post Graduate Evaluation Form for Students and the Level 3 Post Graduate Evaluation Form for Supervisors, will be sent to the students and supervisors about six to nine months following the completion of the training. It is estimated 19,000 respondents for the level 3 Post Graduate Evaluation Form for Students. The number was derived through the analysis of past enrollment and projected schedules of the courses selected to be offered in the future.
It is estimated 9,500 respondents for the level 3 Post Graduate Evaluation Form for Supervisors. This number was derived through the analysis of past student enrollment and projected schedules of the courses selected to be offered in the future.
• Estimation procedure.
Descriptive statistical analysis will be used.
• Degree of accuracy needed for the purpose described in the justification.
This is a consensus and no other sampling procedures are used.
•Unusual problems requiring specialized sampling procedures, and
There are no unusual problems anticipated at this time.
•Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
This is a one time collection.
Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.
Send a Pre-Notification Letter
Level 1 End-of course Evaluation Form will be collected during the course timeframe.
b. A pre-notification email will be sent approximately 1 week prior to the administrative time (six to nine months following the completed training) of the Level 3 Post Graduate Evaluation Form for Students and the Level 3 Post Graduate Evaluation Form for Supervisors.
2. Increase Number of Attempts
a. Level 3 Post Graduate Evaluation Form for Students and the Level 3 Post Graduate Evaluation Form for Supervisors: Multiple emails and phone calls are sent as reminders per the schedule below.
Reminders
a. Level 3 Post Graduate Evaluation Form for Students and the Level 3 Post Graduate Evaluation Form for Supervisors:
Email reminders will be sent at one week intervals for 3 weeks, if a response target of at least 50 percent is not initially met. This approach has proved effective in stimulating high response rate.
Survey Length
a. The survey completion time is minimal at 15 minutes.
Purpose Clearly Described
a. At the beginning of the survey respondents are told about the questionnaire’s purpose, estimated response time, and who is sponsoring and conducting the survey.
It is expected that these measures will help to maintain very high response rates suitable to analysis, but in the event of response rates falling below 50%, a non-response analysis will be performed on the group(s) in question. These analyses will be conducted by using the Questionmark software package and the findings of the analysis will be addressed accordingly.
Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.
Pilot Test
At the beginning of each survey collection period, a pilot test is conducted with no more than 10 persons to discover any potential problems with the survey instrument or administration process. For quality assurance purposes, data from the pilot is reviewed and improvements are made to the survey process as deemed necessary.
Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
Linda S. Pressley 256-321-0112 will collect and analyze the information for the CDP.
From FEMA- Records Management
Person 1: Hakeem Alston
Records Management Division
Office of Management
Federal Emergency Management Agency
Attention: OM-RM
500 C Street, SW
Washington, DC 20472
Office: (202) 202-212-5237
From Center for Domestic Preparedness
Person 2: Linda S. Pressley
Instructional Systems Specialist
Training and Education Division
Center for Domestic Preparedness
61 Responder Drive P.O. Box 1500
Anniston, Alabama 36205
Office: (256) 231-0112
Fax: (256) 847-2092
| File Type | application/msword |
| File Title | Rev 10/2003 |
| Author | FEMA Employee |
| Last Modified By | OCIO PRA Branch |
| File Modified | 2015-12-30 |
| File Created | 2015-03-12 |