SAFETY Act Process Evaluation Surveys

Request for Approval under the “Pretesting Instruments and Procedures for Evaluation, Research, and Evidence Building”

(OMB Control Number: 1601-0037)

TITLE OF INFORMATION COLLECTION: SAFETY Act Process Evaluation Surveys

PURPOSE OF THE COLLECTION:

This data collection is part of the Support Anti-Terrorism by Fostering Effective Technologies Act of 2002 (SAFETY Act) FY24-25 Evaluation Study Plan which seeks to examine the effectiveness of the process and outcomes of the SAFTEY Act. The purpose of this data collection is to conduct surveys with up to 40 SAFETY Act applicants and up to 15 SAFETY Act contract staff in order to better understand applicants’ experiences with submitting SAFETY Act applications and perceptions about the value of SAFETY Act certifications. This data collection will provide recommendations to increase the suitability of the application submission and review process.

TYPE OF COLLECTION ACTIVITY: (Check one)

[x] Survey/questionnaire [ ] Cognitive interview

[ ] Focus group/small group discussion [ ] User testing

[ ] Interviews [ ] Observation/field study

[ ] Other: _______________________

CERTIFICATION:

I certify the following to be true:

1. The collections are voluntary.

2. The collection is low burden for respondents and low-cost for the Federal Government.

3. The collections are non-controversial and do not raise issues of concern to other Federal agencies.

4. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the near future.

5. Personally identifiable information (PII) is collected only to the extent necessary and is not retained.

6. Information gathered is intended to be used for general service improvement and program management purposes.

7. Summaries and aggregate results may be included in public-facing evaluation or similar report.

8. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

Name: Dana Fronczak

Title: Evidence and Evaluation Officer, DHS Science and Technology

All instruments used to collect information must include:

OMB Control No. 1601-0037

Expiration Date: 02/28/2027

DESCRIPTION OF THIS SPECIFIC COLLECTION

To assist review, please provide answers to the following question:

1. Administration of the Instrument

1. How will you collect the information? (Check all that apply)

[ x ] Web-based or other forms of Social Media

[ ] Telephone or other audio technology

[ ] Video

[ ] In-person

[ ] Mail

[ ] Other:

2. Will interviewers or facilitators be used? [ ] Yes [x ] No

2. Description of respondents/participants

The sample will consist of 40 SAFETY Act applicants who submitted SAFETY Act applications, including pre-applications from program office records and up to 15 SAFETY Act contract staff. The evaluation team will recruit applicants who applied from 2017-2023 across different technology types and types of applications and staff working in various roles in order to recruit clients and staff with all types of applications and experience with processing the applications in order to ensure that a sample represents the demographic makeup of the program universe. Although there were separate applicant surveys created and tested based on the type of application due to the small sample size, the same applicant survey will be used for all applicants, regardless of application type.

3. Activity timeframe

The surveys will be collected on a one-time basis from the date of this approval for a 45-day period.

4. Collection procedures

After the sample has been identified, the DHS evaluation team will send an official DHS introduction email to establish the legitimacy of the study and request participation (See Attachment). Within one week the DHS evaluation team will send a follow-up email to provide further information, including a link to the survey for those who wish to participate (See Attachment). The DHS evaluation team will send up to 1 reminder email after 2 weeks to those who are non-responsive (See Attachments).

With the survey, respondents will receive and review the web-based consent form, which includes a description of the data confidentiality disclosure (See Attachment). Based on a pretest with 4 testers, the surveys are each expected to take no more than 20 minutes to complete.

5. Provide the question list (Submit all instruments, instructions, and scripts with this request).

Please see the following attachments: Attachment A-1: SAFTEY Act Applicants Collection Materials, Attachment A-2: SAFETY Act Contract Staff Collection Materials

6. Use and dissemination of the results

Information from the surveys will be used to identify themes and subthemes. These results will be used to inform process improvements for the SAFETY Act application and submission process and released in an evaluation report.

PERSONALLY IDENTIFIABLE INFORMATION

1. Is personally identifiable information (PII) collected? [ ] Yes [x ] No

2. If Yes, will any information that is collected be included in records that are subject to the Privacy Act of 1974? [ ] Yes [ ] No

3. If Yes, has an up-to-date System of Records Notice (SORN) been published? [ ] Yes [ ] No

GIFTS OR PAYMENTS

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [ ] Yes [ x] No

If Yes, describe:

BURDEN HOUR COMPUTATION

The burden hour estimate for client interview data collection is based on pilot testing with 4 testers.

Category of Respondent	Number of Respondents	Participation Time (mins)	Total Burden Hours
SAFETY Act Contract Staff	15	20	5.0
SAFETY Act Applicant	40	20	13.3

BURDEN COST COMPUTATION

The burden cost estimate for client interview data collection is based on Bureau of Labor Statistics May 2023 median wage for all occupations of $31.48 per hour. ¹

Category of Respondent	Number of Respondents	Hourly Rate	Response Time (hours)	Total Estimated Cost
SAFETY Act Contract Staff	15	31.48	0.33	155.83
-SAFETY Act Applicants	40	31.48	0.33	415.54

FEDERAL COST

The Federal cost estimate for survey development, administration and data collection is $25,979 which includes a GS-15 manager and research support by the contractor (Trewon).² The research support by the contractor will be conducted and administered within the resource and budget parameters of the DHS-Trewon Technologies contract and will require no additional resources, staffing, or funding. Federal employee pay rates are based on the 2024 General Schedule pay scale for the Washington, D.C., locality.³

REQUESTED APPROVAL DATE: 09/27/2024

ICR REQUEST CONTACT:

Name: Dana Fronczak

Title: Evaluation Officer, DHS Science and Technology

Telephone Number: 771-216-1812

Email Address: [email protected]

Attachment A:

Attachment A-1: SAFETY Act Applicant Collection Materials

Survey Introduction Email

Survey Invitation Email

Survey Invitation Reminder Email

Survey Data Collection Instrument

Attachment A-2: SAFETY Act Contract Staff Collection Materials

Survey Introduction Email

Survey Invitation Email

Survey Invitation Reminder Email

Survey Data Collection Instrument

Attachment B: Usability Testing

Usability Testing Summary

Moderator’s Guide⁴

Note Taking Guide⁵

1 See See May 2023 National Occupational Employment and Wage Estimates (bls.gov)

2 Based on estimated contractor hours allocated to client interview.

3 https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/23Tables/html/DCB.aspx

4 Moderator’s Guide is from the DHS Usability Testing Kit | Homeland Security

5 Note Taking Guide is from the DHS Usability Testing Kit | Homeland Security