Revised supporting statement - part A_12.12.19

Enhanced Geo-Targeted 3060-XXXX

Wireless Emergency Alerts December 2019

SUPPORTING STATEMENT

A. Justification:

1. The Warning, Alert and Response Network Act, Title VI of the Security and Accountability for Every Port Act of 2006 (120 Stat. 1884, section 602(a), codified at 47 U.S.C. 1201, et seq., 1202(a)) (WARN Act) gives the Federal Communications Commission (Commission) authority to adopt relevant technical standards, protocols, procedures and other technical requirements governing Wireless Emergency Alerts (WEA). The Commission created the WEA system (previously known as the Commercial Mobile Service Alert System) pursuant to the WARN Act to satisfy the Commission’s mandate to promote the safety of life and property through the use of wire and radio communication.

In 2018, the Commission issued a Report & Order to enhance the public safety benefits of WEA by improving the ability of alert originators (e.g., local emergency management offices) to “geo-target” a WEA alert, i.e., direct a WEA alert to a given geographic area (e.g., where there is imminent threat of the loss of life or property).¹ The Commission amended its geo-targeting rules based on the Report & Order to require that Participating Commercial Mobile Service Providers (providers) implement functionality to deliver a WEA alert to 100% of the target geographic area specified by an alert originator with no more than a 0.1 mile overshoot by November 30, 2019, later extending this deadline to December 13, 2019.² The Commission found that there are urgent public safety benefits associated with providers’ expedient compliance with the Commission’s enhanced geo-targeting requirements.³

The Commission submits this information collection request to support a planned test to evaluate the accuracy and performance of providers’ geo-targeting capabilities based on its December 13, 2019 requirements. The test will be conducted in two geographic areas with two alert originating partners of the Commission. The Commission plans to use surveys to collect information to evaluate performance during the test. This collection of information is voluntary. In each of the geographic areas, respondents affiliated with the area’s alert originator (control group) will be asked to complete a preliminary survey delivered via e-mail. This preliminary survey will improve the utility of a subsequent live test survey, which respondents, the control group and other members of the public, will receive via a hyperlink embedded in a WEA test alert. The Commission has developed survey templates, which are available at https://www.fcc.gov/files/weaprelimtestsurveydec2019pdf and https://www.fcc.gov/files/wealivetestsurveydec2019pdf, that are representative of how the surveys will appear on the Commission’s Public Safety Support Center, and seeks OMB approval of these templates as a new information collection. The information sought in this collection is necessary and vital to ensuring that WEA is effective at protecting the life and property of the public.

The collection is authorized under the WARN Act and 47 U.S.C. 151, 154(i), 154(j), 154(o), 218, 219, 230, 256, 301, 302(a), 303(f), 303(g), 303(j), 303(r) and 403.

This information collection affects individuals or households. The Commission is revising the Privacy Impact Assessment (PIA) and modifying the existing System of Records Notice (SORN), FCC/PSHSB-1, FCC Emergency and Continuity Contacts System (ECCS), for the Public Safety Support System to address the personally identifiable information (PII) that will be collected, used, and stored as part of the information collection requirements.

2. The proposed collection will be used by Commission staff to better understand WEA performance, particularly with respect to the accuracy of providers’ geo-targeting capabilities (i.e., the extent to which WEA alerts are reliably received at eligible mobile devices within a target geographic area without impermissible overshoot). The Commission will use this information to further its public safety mission by informing the public of the state of WEA geo-targeting and identifying any current shortfalls in WEA performance that warrant additional efforts. These steps will enhance public safety by encouraging alert originators to use WEA alerts, motivate consumers to act based on alerts, and reduce the potential for subscriber opt-out from, and desensitization to, received alerts.

3. The Commission will collect survey information electronically using a web-based survey platform that the Commission has found to be efficient, user-friendly, and minimally burdensome to respondents in prior collections. Respondents will use their electronic mobile devices to provide information to the Commission efficiently, through the use of selectable answers wherever possible, and without the need for any handwritten responses.

4. This information collection is unique to the geo-targeting aspects of WEA and is not duplicated elsewhere. The Commission is not aware of any already available information collections that capture the extent to which enhanced geo-targeting is effective, particularly based on tests occurring or scheduled to occur after the Commission’s December 13, 2019 deadline. WEA collection OMB Control No. 3060-1113 (“Election Whether to Participate in the Commercial Mobile Alert System”) relates solely to elections by mobile service providers to participate in WEA, and does not relate to geo-targeting performance. The current proposed collection is not duplicative of OMB Control No. 3060-1113.

5. The proposed collection has been carefully designed to minimize the time and amount of data needed for the Commission to achieve its objectives. The Commission’s two alert originating partners will make their employees and other interested participants in its jurisdiction available to serve in the control group, and the Commission will allow the partner to specify the resources (e.g., number of employees) that can participate in the collection without significant economic impact. Moreover, the collection surveys have been designed to include a small number of questions and selectable choices (e.g., using drop-down menus and radio button selection) wherever possible. This will reduce the burden on participants and the economic impact on the Commission’s alert originating partners and other respondents.

6. This collection is a one-time collection spread across two geographic areas that have different terrain types, population densities and wireless propagation characteristics, and thus that provide different performance information. The collection cannot be conducted less frequently. By collecting information in two geographic areas, rather than one, the Commission expects to obtain more useful test results. Namely, it expects to benefit from a larger overall set of responses collected across a more diverse set of testing conditions than would be possible if the collection were performed in one geographic location only. If this collection were not performed, the Commission would not have reliable information on the effectiveness of the Commission’s enhanced geo-targeting requirements and/or on the accuracy of Participating CMS Providers’ geo-targeting capabilities. This would frustrate the Commission’s mandate to promote the safety of life and property.

7. This collection of information is consistent with the guidelines in 5 CFR 1320.5(d)(2). The Commission does not expect this collection to proceed in a manner inconsistent with the specified criteria because the Commission has structured the proposed surveys to limit the frequency and scope of the data requested. Respondents will provide information electronically via a web interface and not through any other written means.

8. The Commission published a 60-day notice in the Federal Register seeking comments from the public on August 20, 2019 (84 FR 43128). Comments were received from The Rehabilitation Engineering Research Center for Wireless Inclusive Technologies (Wireless RERC) and CTIA. We summarize and address these comments below. Our revisions are included in the enclosed revised preliminary and live test survey templates. None of these revisions impacts the burden estimates disclosed in our initial supporting statement.
   
   
   

CTIA Comments and Commission Responses

CTIA generally supports the Commission’s use of the preliminary and live test survey templates to collect information and evaluate the performance of enhanced geo-targeting.

(CTIA Comments at 2.) CTIA requests, however, that we adopt a number of “minor modifications” to the templates to “help enhance the quality, utility, and clarity of the information collected.” (CTIA Comments at 1-2.) These are addressed below.

Questions re: Cellular Service Provider

CTIA believes that questions in the preliminary and live test survey templates related to respondent’s cellular service “should be modified to inquire into not only whether the device had cellular service at that location, but whether it had service with respondent’s provider at that location.” (CTIA Comments at 4-5; emphasis in original.) CTIA believes that this modification would resolve potential ambiguities if another providers’ network (i.e., other than respondent’s providers’ network) is used to transmit a WEA alert during the live test. (CTIA Comments at 5.) The two survey template questions identified by CTIA are:

• Preliminary Survey: “Does your device have cellular service in this location?” which permits responses of “Yes”, “No” or “Not sure”.

• Live Test Survey: “Did your mobile device have cellular service when the alert was sent?” which permits responses of “Yes”, “No” or “Not Sure”.

The Commission has revised its survey templates in response to these comments to clarify the information sought. These revisions are incorporated in the revised preliminary and live test survey templates submitted with this revised Supporting Statement. The questions as revised read:

• Preliminary Survey: “Does your device have cellular service on your service provider’s network in this location?” which permits responses of “Yes (my device had service and was not roaming)”, “No (my device did not have service or was roaming)” or “Not Sure”.

• Live Test Survey: “Did your mobile device have cellular service on your service provider’s network when the alert was sent?” which permits responses of “Yes (my device had service and was not roaming)”, “No (my device did not have service or was roaming)” or “Not Sure”.

These revisions address CTIA’s comments by clarifying for respondents how to respond if they are not using their own providers’ network when filling out their responses. These revisions do not impact the burden estimates contained in our initial supporting statement.

Questions re: Type of Cellular Service

CTIA notes that the Commission’s live test survey asks respondents to provide information on what type of service their mobile device had during the test, permitting answers of “4G”, “3G”, “2G” and “Other/ Not sure”. (CTIA Comments at 5.) CTIA, however, believes that “‘type of service’ has no bearing on quality, utility, and clarity of information collected.” (CTIA Comments at 5.) CTIA urges us to remove this question from its survey template. (CTIA Comments at 5.)

We disagree that the survey templates should be modified to remove this question. Based on our experience, we believe that it is possible that the successful reception of WEA alerts at respondents’ devices will vary depending on the type of service used (2G, 3G or 4G), particularly if respondents are actively using their devices at the same time that alerts are received. We believe that including this question in the live test survey template will allow us to test this hypothesis. Moreover, we believe that this question will not be unduly burdensome as it will be readily understood and answerable by survey respondents. Therefore, we do not propose any amendments at this time.

Questions re: Make and Model and Location Information

CTIA requests that the Commission modify the following the following prompt of the preliminary and live tests survey templates: “[w]hat is the make and model of your mobile device (e.g., ‘Samsung Galaxy S10,’ ‘Apple [i]Phone X.’) If you do not know, leave this field blank.” Specifically, CTIA believes that we should require survey respondents to provide an answer to this question and not allow them to optionally leave this field blank. (CTIA Comments at 3.) Similarly, CTIA requests that that we remove the option for survey respondents to optionally leave blank certain prompts in our survey templates related to location information, i.e., the prompt “[w]here will you be located at the time and day of the test (If you know, otherwise leave these fields blank)?” in our preliminary survey template and “[w]here were you when you received the alert . . . If you did not receive the alert, leave this field blank” in our live test survey template. (CTIA Comments at 4.) CTIA believes that collecting this information from survey respondents is critical to evaluating test results. (CTIA Comments at 3-4.)

We agree with CTIA that meaningful responses to the questions identified are highly relevant and important information. We disagree, however, that the surveys should be modified to prevent respondents from optionally leaving these fields blank. CTIA’s approach would force respondents to provide a response to these questions even when they do not know or are unsure of an answer. This would make it hard or impossible for the Commission and its partners to separate reliable survey responses from unreliable ones, and ultimately lower the reliability of our test results analysis. Moreover, we expect that the control group will be trained on the information sought by these questions and thus that these fields will rarely be left blank by control group respondents in practice. Therefore, we do not propose any amendments at this time.

Questions re: Operating System

CTIA believes that we should modify our preliminary and live test survey templates “to request not only the respondent’s operating system, but also their device’s current software version.” (CTIA Comments at 3-4.) CTIA believes that the ability of devices to support enhanced geo-targeting may vary between different software versions on the same operating system. (CTIA Comments at 3-4.)

We acknowledge CTIA’s view that test results may vary between different software versions on the same operating system. We disagree, however, that the survey templates should be modified to ask for respondents’ software version information. Based on the Commission’s statistical analyses in Part B (“Collections of Information Employing Statistical Methods”) of this revised supporting statement, we do not believe that the sample size of the control group is large enough to allow us to draw statistically significant conclusions at the software versioning level. Therefore, we do not propose any amendments at this time. However, we will take this comment under advisement for potential future WEA testing efforts that may involve still large samples sizes.

Request for Sharing of Results

CTIA requests that the Commission share the results of the preliminary and live test surveys with Participating Commercial Mobile Service Providers. (CTIA Comments at 5.) As noted in response to question 16 below, the Commission will determine whether and how to publish results after the testing is complete.

Summary of Wireless RERC Comments

Wireless RERC is supportive of the FCC’s initiative to collect data on the geo-targeting capabilities of WEA messages and notes that the proposed collection purpose would assess a “critical measure” of the effectiveness of WEA messages. (Wireless RERC Comments at 1-2.) Wireless RERC “recommend[s] that the FCC, at a later date, consider collecting data on factors impacting WEA efficacy for people with disabilities” and notes that such factors may include “the level of awareness of WEA, the extent to which WEA-enabled devices are diffused amongst the population, the accessibility of the message to people with disabilities, and recipients having a favorable behavioral response to the messages.” (Wireless RERC Comments at 2.) Wireless RERC offers subject matter expertise and proposes that proposes that "FCC staffers to contact [it] regarding future surveying of the disability community.” (Wireless RERC Comments at 2.)

Commission Response to Wireless RERC Comments

Wireless RERC’s comments relate to proposed “future surveying of the disability community” and do not provide specific proposals for the current information collection. Because the comments are outside the scope of the current information collection, the Commission does not propose any amendments to the current information collection based on Wireless RERC’s comments. The Commission has considered these comments and we will take them under advisement as they relate to any of our future WEA assessment efforts.

Information from Other Relevant Stakeholders

In addition, the Commission has solicited information from relevant stakeholders, including alert originators, via a Public Notice regarding the appropriate scope of the control group and other testing parameters.⁴ The Commission has received and considered responsive information provided by the New York City Emergency Management Department, the Harris County, TX Office of Homeland Security and Emergency Management, Philadelphia’s Office of Emergency Management (Philadelphia), and the Los Angeles Emergency Management Department (LA).  

9. The Commission will not provide any payment or gift to respondents.

10. As noted in Question 1, this information collection affects individuals or households. The Commission is revising the Privacy Impact Assessment (PIA) and modifying the existing System of Records Notice (SORN), FCC/PSHSB-1, FCC Emergency and Continuity Contacts System (ECCS), for the Public Safety Support System to address the personally identifiable information (PII) that will be collected, used, and stored as part of the information collection requirements.

11. There are no questions of a sensitive nature involved with this collection of information. Moreover, as noted in Question 1, the Commission is revising the Privacy Impact Assessment (PIA) and modifying the existing System of Records Notice (SORN), FCC/PSHSB-1, FCC Emergency and Continuity Contacts System (ECCS), for the Public Safety Support System to address the personally identifiable information (PII) that will be collected, used, and stored as part of the information collection requirements.

12. Burden estimates are as follows:

Preliminary Survey for Control Group #1 (corresponding to Geographic Area #1)

• Number of Respondents: 1,000.

• Frequency of response: One-time reporting requirement.

• Total Number of Responses: 1,000 respondents × 1 response/respondent = 1,000 total responses.

• Average response time per response: 0.25 hours.

• Total Burden Hours: 1,000 responses × 0.25 hours/response = 250 hours.

Preliminary Survey for Control Group #2 (corresponding to Geographic Area #2)

• Number of Respondents: 1,000.

• Frequency of response: One-time reporting requirement.

• Total Number of Responses: 1,000 respondents × 1 response/respondent = 1,000 total responses.

• Average response time per response: 0.25 hours.

• Total Burden Hours: 1,000 responses × 0.25 hours/response = 250 hours.

Method of estimation of burden: We estimate that the total number of control group participants will be 2,000 (i.e., 1,000 participants from each of control groups #1 and #2). We believe that this represents the largest size of the control group that the Commission’s alert originating partners would make available based on the Commission’s experience working with alert originators and information received in response to the 7/11/19 Public Notice.

The time estimate is based on sub-estimates that the time needed for data entry and submission is 10 minutes and the time needed to gather the associated information is five minutes (for a total of 15 minutes or 0.25 hours). In making this time estimate, we have considered that respondents will enter information electronically and that any information to be gathered is available directly on the mobile device that will be used by the respondent to provide their response.

Preliminary Survey “In-House” Costs: The Commission estimates the hourly wage of a full-time employee (and other control group members) who will be submitting this information as $30/hour, inclusive of overhead and fringe benefit costs. This estimate is based the Commission’s analysis of 2018 Bureau of Labor Statistics nationwide average hourly wages and salaries for technicians and other professionals engaged in field work.⁵ Therefore, the in-house costs to the respondents are as follows:

500 total burden hours × $30/hour = $15,000 total annual “in-house” costs.

Live Test Survey for Control Group #1 (corresponding to Geographic Area #1)

• Number of Respondents: 6,000.

• Frequency of response: One-time reporting requirement.

• Total Number of Responses: 6,000 respondents x 1 response/respondent = 6,000 total responses.

• Average response time per response: 0.25 hours.

• Total Burden Hours: 6,000 responses × 0.25 hours/response = 1,500 hours.

Live Test Survey for Control Group #2 (corresponding to Geographic Area #2)

• Number of Respondents: 6,000.

• Frequency of response: One-time reporting requirement.

• Total Number of Responses: 6,000 respondents x 1 response/respondent = 6,000 total responses.

• Average response time per response: 0.25 hours.

• Total Burden Hours: 6,000 responses × 0.25 hours/response = 1,500 hours.

Method of estimation of burden: This estimate is based on our estimate of 2,000 control group respondents, identified above, and an additional 10,000 respondents from the public across the two live tests. The true number of respondents from the public in each geographic area will depend on the number of individuals that happen to be present in and around the geo-targeted area when the WEA test alert is transmitted, that have opted-in to receive WEA test alerts, that actually receive the test alert and that elect to respond to the link embedded in the alert and complete the live test survey. Based on experience, the Commission expects this number to be no more than 10,000 respondents (i.e., based on a belief that there will be no more than 5,000 respondents in each geographic area), in addition to the control group.

The time estimate is based on sub-estimates that the time needed for data entry and submission is 10 minutes and the time needed to gather the associated information is five minutes (for a total of 15 minutes or 0.25 hours). In making this time estimate, we have considered that respondents will enter information electronically and that any information to be gathered is available directly on the mobile device that will be used by the respondent to provide his or her response.

Live Test Survey “In-House” Costs: As noted above, the Commission estimates the hourly wage of the estimated 2,000 employees (and other control group members) affiliated with the alert originators who will be submitting this information as $30/hour, inclusive of overhead and fringe benefit costs. The Commission estimates the effective hourly wage of the average member of the public who responds to the survey to also be $30, inclusive of overhead and fringe benefit costs. This estimate based the Commission’s analysis of 2018 Bureau of Labor Statistics nationwide average hourly wages and salaries for all occupations.⁶ Therefore, the in-house costs to the respondents are as follows.

• (2,000 responses from preliminary survey + 10,000 responses from live test survey) × 0.25 hours/response = 3,000 burden hours

• 3,000 burden hours × $30/hour = $90,000 “in-house” costs.

Cumulative Totals

• Total Respondents Annually: 2,000 (participants for preliminary survey) + 10,000 (additional participants for live test survey) = 12,000 respondents;

• Total Annual Responses: 2,000 (preliminary survey) + 12,000 (live test survey) = 14,000 responses;

• Total Annual Hours Burden: 2,000 responses (preliminary survey) × 0.25 hr./response + 12,000 responses (live test survey) × 0.25 hr./response = 3,500 hours;

• Total Annual “In-House” Costs: 3,500 hours x $30/hr. = $105,000.

13. There is no outside cost to the respondents.

14. The Commission does not expect to incur costs beyond the normal labor costs for staff.

15. This is a new information collection resulting in a program change/increase to the total number of respondents of 12,000, total annual responses of 14,000 and total annual burden hours of 3,500 hours. These estimates will be added to OMB’s Active Inventory

16. The Commission may elect to publish a summary of results from its tests. If it does so, the Commission expects that the published results will be accessible from the Commission’s website, including in the dockets of related proceedings, PS Docket Nos. 15-91 and 15-94. A Commission summary may include some or all of (a) tabulations indicating the number of survey results received by provider, device operating system, geographic area and sub-area and participant type (control group and non-control group), (b) cumulative information categorizing the responses received to individual survey questions, (c) success rates and measures of Bayesian statistical significance, and (d) key trends affecting providers’ geo-targeting performance. Additional details on the Commission’s data collection and analysis are provided in section B of this Supporting Statement, below.

The Commission expects that the collection in geographic area #1 will occur on two days (corresponding to the preliminary test and live test, respectively) and that the collection in geographic area #2 will occur on two days (again, corresponding to the preliminary test and live test, respectively). The Commission will determine specific test dates together with its alert originating partners and any of its publication dates after the testing is complete.

17. The Commission is requesting a waiver of the requirement to display the OMB expiration date on the surveys because that would require updating each time this collection was submitted to OMB for review and approval. The Commission displays the OMB expiration date, title and OMB control number in 47 CFR 0.408 of the Commission’s rules.

18. There are no exceptions to the Certification Statement.

B. Collections of Information Employing Statistical Methods:

The collections of information does employ Statistical Methods. The Commission has attached the Statistical Methodology - Part B in this submission to OMB.

1 See Wireless Emergency Alerts; Amendments to Part 11 of the Commission's Rules Regarding the Emergency Alert System, PS Docket Nos. 15-91 and 15-94, Second Report and Order and Second Order on Reconsideration, 33 FCC Rcd 1320, 1324-25, para. 6 (2018); see also Federal Communications Commission, Wireless Emergency Alerts; Emergency Alert System, 83 Fed. Reg. 8619, 8623 (Feb. 28, 2018) (announcing a Nov. 30, 2019 amendment to 47 CFR § 10.450); see also New Enhancements to Wireless Emergency Alerts Will Be Available on December 13, 2019, Public Notice, DA 19-1208 (PSHSB November 25, 2019) (extending the deadline for compliance from Nov. 30, 2019 to Dec. 13, 2019).

2 See id.

3 See id.

4 FCC Seeks Alert Originators to Participate in WEA Test, Public Notice, DA 19-551 (PSHSB July 11, 2019) (7/11/19 Public Notice).

5 Bureau of Labor Statistics, May 2018 National Occupational Employment and Wage Estimates United States (May 2018), https://www.bls.gov/oes/current/oes_nat.htm#00-0000.

6 Id.

7