Submission A

Request for Approval under the “Generic Clearance for the Collection of Routine Customer Feedback” (OMB Control Number: 1670-0027)

:

TITLE OF INSTRUMENT:

CISA Exercise Participant Survey

PURPOSE:

The CISA Exercises program, established as a sub-division and executive sponsor for exercises in July 2019 as part of the CISA 2020 realignment, combines legacy programs from CISA’s Infrastructure Security Division (ISD) and Cybersecurity Division (CSD). CISA Exercises conducts cyber and physical exercises with government, sector, and international critical infrastructure owners and operators to enhance security and resilience of critical infrastructure. CISA Exercises also shares trends that inform CISA-wide development of risk mitigating resources. Exercise products include Tabletop Exercise Packages or National Exercises and associated planning and coordination meetings for each. In order for CISA Exercises to understand behavior change of an organization and the impact of the exercise, the study team will collect data from exercise participants after they have participated in an exercise, to explore what actions they have taken and what barriers they have encountered.

The CISA Exercises study team will use the surveys and in-depth interviews to measure the degree in which exercise customers are implementing recommended corrective actions identified through the exercise process, and, in cases where they are not, to better understand the major factors limiting corrective action implementation. The long-term outcome of increased preparedness and resilience is dependent upon the intermediate-term outcome of customers implementing corrective actions to address planning and capability gaps identified through the exercise process. The results of the evaluation will inform CISA Exercises, other CISA capability-building programs, and CISA at-large about how best to leverage and improve upon its capability-building approaches.

DESCRIPTION OF RESPONDENTS:

Survey participants will include state, local, tribal, territorial, and private sector community partners who have participated in a CISA exercise.

TYPE OF COLLECTION: (Check one)

[ ] Customer Comment Card/Complaint Form [X] Customer Satisfaction Survey

[ ] Usability Testing (e.g., Website or Software [ ] Small Discussion Group

[ ] Focus Group [ ] Other: ______________________

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name: /s/ Mia Bruce and Cammie Chiarella

To assist review, please provide answers to the following question:

Personally Identifiable Information:

1. Is personally identifiable information (PII) collected? [X] Yes [ ] No

2. If yes, is the information that will be collected included in records that are subject to the Privacy Act of 1974? [ ] Yes [X] No

3. If Applicable, has a System or Records Notice been published? [ ] Yes [X] No

Gifts or Payments:

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [ ] Yes [X] No

BURDEN HOURS

Instrument	Total Annual Number of Respondents	Number of Responses per Respondent	Average Burden per Response (hours)	Total Time Burden (hours)	Average Hourly Compensation Rate	Total Annual Cost
Web Survey only	70	1	0.17	11.67	$85.95	$1,002.79
Web Survey & In-depth Interview	30	1	1.17	35	$85.95	$3,008.38
Total	100			46.67		$4,011.18

To estimate the burden associated with completing the data collection activities, CISA Exercises uses the loaded average hourly compensation rate for General and Operations Managers of public and private sectors. CISA Exercises used Bureau of Labor Statistics (BLS) data to estimate the average hourly wage rate for General and Operations Managers in public and private sectors of $60.45¹. To account for benefits, DHS multiplies this average hourly wage rate by a compensation factor of 1.4219², which is the ratio of total compensation to salaries and wages. The fully loaded average hourly compensation rate is $85.95.³

To estimate the labor cost for the respondents to complete the data collection activities, CISA Exercises multiplies the total annual time burden by the loaded average hourly compensation rate. As shown in Table A.1, CISA Exercises estimates the labor cost to complete just the web survey to $1,002.79. The estimated labor cost to complete both the web survey and in-depth interview is $3,008.38. In total, CISA Exercises estimates the elements of this information collection would result in 46.67 burden hours and $4,011.18 in annual labor costs.

FEDERAL COST: The estimated total cost to the Federal government is $99,759.97over a 2-year period, or $49,879.99 on an annualized basis, based on the expected cost of the government contract used to conduct this collection.

STATISTICAL METHOD:

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents

1. Do you have a customer list or something similar that defines the universe of potential respondents and do you have a sampling plan for selecting from this universe? [X] Yes [ ] No

If the answer is yes, please provide a description of both below (or attach the sampling plan)? If the answer is no, please provide a description of how you plan to identify your potential group of respondents and how you will select them?

CISA Exercises conducts between 80 to 100 exercises a year. Each exercise can include multiple people from the customer organization as well as individuals from partner organizations. The CISA Exercises study team will first select exercises that occurred 5 to 8 months before the planned data collection start, so that enough time has passed for customers to have implemented many of the recommendations outlined in the After-Action Report. Exercises with federal agencies will be excluded. Within the selected exercises, the study team will select up to 5 key participants for the web survey based on their level of involvement in the exercises process and likely knowledge of organizational change. Respondents will be sent an email invitation to complete the survey with an individual Survey Monkey link. Following the web survey, a purposive sample of up to 30 survey respondents will be contacted for the in-depth interview.

Administration of the Instrument

1. How will you collect the information? (Check all that apply)

[X] Web-based or other forms of Social Media

[X] Telephone

[ ] In-person

[ ] Mail

[ ] Other (email response)

For use of Survey Monkey, CISA will ensure that respondents IP addresses are not collected by ensuring that the Survey Monkey settings are updated to opt out of collecting this information.

2. Will interviewers or facilitators be used? [X] Yes, for telephone or in-person distribution [ ] No

Please make sure that all instruments, instructions, and scripts are submitted with the request.

1 BLS. Occupational Employment Statistics. May 2020. www.bls.gov/oes/current/oes111021.htm

2 BLS. Employer Costs for Employee Compensation – December 2020, released March 18, 2021. Private industry workers. https://www.bls.gov/news.release/pdf/ecec.pdf

The compensation factor of 1.4219 is estimated by dividing total compensation ($36.23) by wages and salaries ($25.48).

3 $85.95 = $60.45 × 1.4219.

4