1670-0048 SNS Testing Generic Clearance Submission

Department of Homeland Security OMB Control Number: 1670-0048

Cybersecurity and Infrastructure Security Agency OMB Expiration Date: 11/30/2025

Emergency Communications Division

SAFECOM Nationwide Survey (SNS) Testing

TITLE OF INSTRUMENT:

SAFECOM Nationwide Survey Questions and Feedback Form

PURPOSE:

To meet the statutory requirements of 6 U.S.C. § 573, the Department of Homeland Security (DHS) Cybersecurity and Infrastructure Security Agency (CISA) Emergency Communications Division (ECD) is mandated to conduct the SAFECOM Nationwide Survey (SNS) every 5 years to assess evolving capability needs and gaps and track progress against policy initiatives; status of strategic plans; and major industry or market shifts affecting the emergency communications capability. The last SNS was conducted in late 2017 through early 2018. To ascertain this information, the SNS will deploy three similar surveys through Qualtrics or alternate survey forms to various emergency response disciplines at each level of government - federal, state/territorial, local/tribal. The survey will solicit responses regarding issues affecting the public safety community to determine a jurisdiction’s level of operability, interoperability, and continuity and thus their overall emergency communications capability level. CISA ECD will analyze the data collected from this general survey to identify major gaps and themes affecting emergency communications across levels of government. Additionally, this analysis informs future iterations of the Nationwide Baseline Communications Assessment (NCBA) and National Emergency Communications Plan (NECP). The data is shared with all stakeholders that have a role in emergency communications.

Testing of the survey prior to national launch allows CISA ECD to gather insight on whether respondents can easily understand and answer questions, the organization of the document, the estimated completion time, and whether and where there is an onset of survey fatigue. The findings are then used to make modifications to the questions or survey platform, as needed, prior to launching the survey

DESCRIPTION OF RESPONDENTS:

Survey testers will consist of approximately 75 participants representing federal, state, local, tribal, and territorial public safety organizations.

TYPE OF COLLECTION: (Check one)

[ ] Customer Comment Card/Complaint Form [ ] Customer Satisfaction Survey

[ X] Usability Testing (e.g., Website or Software) [ ] Small Discussion Group

[ X] Focus Group [X] Other: ___Feedback Form__

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name: Charlee Hess [email protected]

To assist review, please provide answers to the following question:

Personally Identifiable Information:

1. Is personally identifiable information (PII) collected? [ ] Yes [X] No

2. If yes, is the information that will be collected included in records that are subject to the Privacy Act of 1974? [ ] Yes [X] No

3. If Applicable, has a System or Records Notice been published? [ ] Yes [X] No

Gifts or Payments:

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [ ] Yes [X] No

BURDEN HOURS

CISA ECD is seeking approximately 75 participants from various levels of governments and disciplines to review the SNS and test the online tool before it is released. There will be 25 federal testers, 20 state/territorial testers, and 30 local/tribal testers. It is anticipated that each tester will spend approximately three hours to complete SNS testing: 1 hour for a kickoff meeting to receive instructions and review objectives; 1 hour to complete the assignment; and 1 hour to participate in a facilitated feedback session to obtain inputs on the ease in which they understood and answered questions, the organization of the document, the estimated completion time, and whether and where the onset of survey fatigue set in. The following burden has been calculated for the 50 non-federal testers.

Survey	No. of Testers	Participation Time	Burden	Annual Burden Cost
SNS Testing – Non-Federal	50	3 Hours (180 minutes)	150 hours/year	$5,881.50 ($117.63 per person)

The above Non-Federal Average Hourly Wage Rate is the May 2022 Bureau of Labor Statistics average wage for “All Occupations” of $28.01 times the wage rate benefit multiplier of 1.4 (to account for fringe benefits) equaling $39.21. The selection of “All Occupations” (for example) was chosen as the expected respondents for this collection could be expected to be from any occupation.

FEDERAL COST: The estimated annual cost to the Federal government is $6,372.

It is estimated that 25 government employees at a GS-14, Step 1 will spend approximately 0.144% of their time (2080 annual working hours x 1% = 3 hours) annually to review test questions and provide feedback. For a total of 75 hours annually (25 staff x 3 hours each).

Using the FY22 (Washington-Baltimore-Northern Virginia) GS pay scale, the fully-loaded wage rate for a GS14, Step 1 is $ 75.40 ($ 126,233 annual salary/2080 = $60.69 base wage rate x 1.4 benefit multiplier = $ 84.96 fully-loaded wage rate).

The annual government cost is estimated to be $6,372 (75 hours x $84.96 = $6,372).

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents

1. Do you have a customer list or something similar that defines the universe of potential respondents and do you have a sampling plan for selecting from this universe? [X] Yes [ ] No

If the answer is yes, please provide a description of both below (or attach the sampling plan)? If the answer is no, please provide a description of how you plan to identify your potential group of respondents and how you will select them?

CISA ECD will conduct test groups focused on specific aspects of the survey and organized by level of government (three federal, two state/territorial, and two local/tribal) focused on specific aspects of the survey. Each test group will consist of representatives from the five disciplines of fire and rescue, law enforcement, emergency medical services (EMS), emergency communications centers/public safety answering points (ECC/PSAP), and emergency management. CISA ECD is targeting 25 federal testers, 20 state/territorial testers, and 30 local/tribal testers. The local/tribal level of government will have more participants than the other levels of government given that they represent the largest population anticipated to complete the survey.

There will three categories for testing – Group A, Group B, and Group C. Each group is looking at a different perspective of the survey content or tool. All three levels of government will participate in Test Groups A and B. However, only federal testers will participate in Test Group which will review the alternate survey forms. This results in the need for a minimum of 50 state, territorial, tribal, and local testers. Table 1 summarizes the scope of each test group.

Table 1. Test Group Characteristics

	Focus Areas	Testing Mechanism	Number of Participants	Desired Composition/Expertise of Testers1
Test Group A (3 Groups)	Respondent instructions Questions Answer instructions Answer options Question placement	Online Tool	10 federal 10 state/territorial 15 local/tribal (35 total)*	Fire and Rescue (3-5) Law Enforcement (3-4) EMS (1-3) ECC/PSAP (1-3) Emergency Management (1-3)
Test Group B (3 Groups)	Real-time survey experience Respondent fatigue Completion time General observations	Online Tool	10 federal 10 state/territorial 15 local/tribal (35 total)	Fire and Rescue (3-5) Law Enforcement (3-4) EMS (1-3) ECC/PSAP (1-3) Emergency Management (1-3)
Test Group X (Federal only)	Alternate survey forms	Paper and online pdf versions	5 Federal	ECD Staff

Administration of the Instrument

1. How will you collect the information? (Check all that apply)

[X] Web-based or other forms of Social Media

[ ] Telephone

[ ] In-person

[X] Mail

[X] Other (virtual feedback session and voluntary comment form)

The test environment will, as closely as possible, replicate the actual assessment environment for all levels of government (e.g., federal, state/territorial, and local/tribal). For the online version, testers will be asked to test online using Qualtrics. A link to the survey can be found at: https://cisaecd.gov1.qualtrics.com/jfe/form/SV_9SLCcO7oycNTpH0.

The alternate forms will be tested using paper copies and Adobe fillable forms. Testers completing the hard copies will be provided with a pre-paid mailing envelope to submit the completed tests to CISA.

For use of Qualtrics, CISA will ensure that tester IP addresses are not collected by ensuring that the Qualtrics settings are updated to opt out of collecting this information.

CISA ECD will conduct an integrated kickoff meeting with testers from all levels of government for each major group (A, B, and X) during which the objectives and scope of questions will be covered along with instructions for providing input. Each group will be provided with a voluntary comment form that testers can use to take notes as they review the survey content. CISA ECD will conduct 60 min facilitated feedback sessions with each sub-group to capture overarching concerns or challenges.

2. Will interviewers or facilitators be used? [X] Yes, for telephone or in-person distribution [ ] No

Please make sure that all instruments, instructions, and scripts are submitted with the request.

CISA ECD will use internal staff or contract support to lead virtual feedback sessions through Teams.

1 Weighting is based on the percentage of population listed in the Paperwork Reduction Act Generic Clearance package. Based on total number of agencies /departments ECD used were data set entries listed by the Bureau of Indian Affairs, the National Public Safety Information Bureau, and HIFLD plus the 1,913 state/territorial-level disciplinary responses, the percentage of population by discipline is: Law Enforcement: 28%; Fire and Rescue: 47%; EMS: 8%; ECC/PSAP: 8%; and Emergency Management: 9%.

5