2021-07-02 - BEYOND and PSP - Supporting Statement B - Final

Supporting Statement

Unmanned Aircraft Systems (UAS) BEYOND and Partnership for Safety Plan (PSP) Programs

B. Statistical Methods

The purpose of the Unmanned Aircraft Systems (UAS) BEYOND and Partnership for Safety Plan (PSP) programs is for the Federal Aviation Administration (FAA) to work with state, local and tribal (SLT) governments (BEYOND) and private UAS operators (PSP) to work toward full, safe integration of UAS into the national airspace system (NAS). There are eight SLT governments in the BEYOND program and seven industry participants in the PSP program. The programs have multiple data collections that serve different purposes to help the FAA achieve the goals.

First, there are narrative reports that will provide qualitative, non-statistical data that will inform the FAA of operational trends, highlight successes and failures and their causes, and describe challenges and lessons learned. These narrative reports are meant to inform the FAA of progress being made, to identify where there are challenges or gaps in understanding, and to help the FAA make policy and resource decisions. The data will be supplied by the eight SLT governments in the BEYOND program, and the seven industry operators in the PSP program. The collection instruments include:

1. PSP Quarterly Reports

2. BEYOND Semi-Annual Reports

3. BEYOND Final Reports

4. PSP Final Reports

5. BEYOND Program Withdrawal Reports

Second, there are operational data submissions which will provide both quantitative and qualitative information about the program participants’ aircraft, flights, corrective maintenance actions, off-nominal flight events, and UAS testing activities. The submissions are not statistical in nature but are designed to supply data that will help inform policy and standards related to UAS pilots flying their aircraft beyond their visual line of sight. The data will be supplied by the eight SLT governments in the BEYOND program, and the seven industry operators in the PSP program. The collection instruments include:

6. UAS Characteristics Reports

7. UAS Monthly Operational Flight Reports

8. UAS Maintenance Reports

9. UAS Flight Anomaly Reports

10. UAS Test Data Reports

Third, there are societal and economic data submissions that will provide qualitative and quantitative data regarding the potential societal and economic impacts of participant UAS operations. These are non-statistical reports that provide insight into the types of societal and economic benefits or detriments the participants’ UAS operations are having on their communities and their business operations compared to non-UAS (legacy) operations. These submissions will help to inform FAA policy and decision-making toward integrating different types of operations into the NAS, and to inform discussions with the public on the potential benefits of using UAS. The data will be supplied by the eight SLT governments in the BEYOND program, and the seven industry operators in the PSP program. The collection instruments include:

11. Legacy Societal and Economic Data Reports

12. UAS Societal and Economic Data Reports

Fourth, there is a community engagement component to the programs which includes two separate, but related, collections. One of these is the reports that will provide qualitative and quantitative data regarding the participants’ community outreach activities, including the types of activities, the targeted audiences, and the types and quantity of feedback received. The reports will include any raw data collected using questionnaires or surveys. These submissions are not statistical in nature, but will provide valuable information to the FAA that will inform community engagement best practices and lessons learned which can be shared with the public. The data will be supplied by the eight SLT governments in the BEYOND program, and the seven industry operators in the PSP program. The collection instruments will include:

13. Community Engagement Reports

The other piece of the community engagement component is the Community Engagement Tool, which will include a potential burden on the general public. The Community Engagement Tool was developed by the FAA as an optional aid for program participants that includes general and operation-specific questions related to UAS operations. The BEYOND and PSP participants may choose to use any of the pertinent sample survey questions when developing their own questionnaires to survey the general public in their communities. The tool includes questions on general knowledge of UAS and sentiments toward potential benefits or drawbacks of UAS operations. The Community Engagement Tool is intended to help the program participants develop simple questionnaires with no intended utility other than gaining insight into the general public’s experience with, and opinions of, UAS operations. No statistical analysis or inference will be performed other than tabulation of responses. Results may be used to help develop lessons learned or best practices for other UAS stakeholders pertaining to community engagement activities.

14. Community Engagement Tool

1. Describe the potential respondent universe.

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Report/ Form	Affected Public	Frequency	Number of Respondents	Total Number of Responses	Estimated Average Burden Per Response (hours)	Estimated Total Annual Burden (hours)
Narrative Reports
PSP Quarterly Reports	Business or other for-profit, PSP participants only	Quarterly	7.00	28.00	80.00	2,240.00
BEYOND Semi-Annual Reports	State, Local or Tribal Government, BEYOND participants only	Semi-Annually	8.00	16.00	80.00	1,280.00
PSP Final Reports	Business or other for-profit, PSP participants only	One-Time Submission	7.00	2.33	40.00	93.20
BEYOND Final Reports	State, Local or Tribal Government, BEYOND participants only	One-Time Submission	8.00	2.67	40.00	106.80
BEYOND Program Withdrawal Reports	State, Local or Tribal Government, BEYOND participants only	One-Time Submission	8.00	2.67	40.00	106.80
Narrative Reports Sub-Totals			38.00	51.67	74.06	3826.80
Systems and Operations Data
UAS Monthly Operational Flight Reports	Business or other for-profit – PSP participants only, and State, Local or Tribal Government – BEYOND participants only	Monthly	15.00	180.00	1.00	180.00
UAS Maintenance Reports	Business or other for-profit – PSP participants only, and State, Local or Tribal Government – BEYOND participants only	Monthly	15.00	180.00	1.00	180.00
UAS Test Data Reports (optional)	Business or other for-profit – PSP participants only, and State, Local or Tribal Government – BEYOND participants only	Ad hoc	15.00	15.00	0.08	1.25
UAS Anomaly Reports	Business or other for-profit – PSP participants only, and State, Local or Tribal Government – BEYOND participants only	On Occasion – Assuming 10 annually per participant	15.00	150.00	1.00	150.00
UAS Aircraft Characteristics Submissions	Business or other for-profit – PSP participants only, and State, Local or Tribal Government – BEYOND participants only	On Occasion – for each New Aircraft – Assuming average 25 annual submissions	15.00	375.00	0.20	75.00
Systems and Operations Data Sub-Totals			75.00	900.00	0.65	586.25
Societal and Economic Data
Legacy Societal and Economic Data Reports (PSP participants)	Business or other for-profit – PSP participants only	Quarterly	7.00	28.00	1.00	28.00
UAS Societal and Economic Data Reports (PSP participants)	Business or other for-profit – PSP participants only	Quarterly	7.00	28.00	1.00	28.00
Legacy Societal and Economic Data Reports (BEYOND participants)	State, Local or Tribal Government – BEYOND participants only	Semi-Annually	8.00	16.00	1.00	16.00
UAS Societal and Economic Data Reports (BEYOND participants)	State, Local or Tribal Government – BEYOND participants only	Semi-Annually	8.00	16.00	1.00	16.00
Societal and Economic Data Sub-Totals			30.00	88.00	1.00	88.00
Community Engagement
Community Engagement Data Reports (PSP participants)	Business or other for-profit – PSP participants only	Quarterly	7.00	28.00	2.00	56.00
Community Engagement Data Reports (BEYOND participants)	State, Local or Tribal Government – BEYOND participants only	Semi-Annually	8.00	16.00	2.00	32.00
Community Engagement Tool (optional)	Individuals or Households	On Occasion	175,005.00	175,005.00	0.17	29,167.00
Community Engagement Sub-Totals			175,020.00	175,049.00	0.17	29,255.00
Totals			175,163.00	176,088.67	0.19	33,756.05

For collection instruments one (1) through 13, the respondents will include the eight (8) BEYOND state, local and tribal government lead participants and/or the seven (7) PSP industry participants. We expect all of the participants to submit the data based upon the reporting frequencies described in the chart above. None of these data collections are meant to be statistical in nature, but to be specific to the aircraft and operations for each participant. The data will be aggregated when possible for data analysis purposes as one of the tools to inform potential policy and/or decision making.

The final collection instrument, number 14, is a Community Engagement Tool. Each of the state, local and tribal government participants in the BEYOND program have submitted community engagement plans related to the concepts of operation for themselves and their industry partners. Those plans show that most public engagement will be targeted to local populations or stakeholders directly affected by the UAS operations, meaning use of Community Engagement Tool questions will be relatively limited for now as operations have been limited in number and scope, and increase yearly as operational concepts are proven and expanded to new areas and from more rural areas to suburban and urban areas over time. Most outreach will be in the form of small events such as town halls or communications directly with local neighborhoods. We estimate that on average, the questions will reach no more than 5,000 people per year per participant the first year, and perhaps double each year thereafter. That would bring the total to 35,000 members of the public potentially asked to answer the questions per participant over the course of three years, annualized to 11,667 per participant per year. We will assume a 100% response rate to determine the maximum burden on the public. There are eight participants in the BEYOND program and seven in the PSP program, which would bring the total estimated respondents to 175,005 per year.

The community engagement tool is intended to help the participants develop simple questionnaires with no intended utility other than gaining insight on the opinions and experience of the general public regarding UAS operations. No statistical analysis or inference will be performed, other than tabulation of responses. Results may be used to help develop lessons learned or best practices worksheets for stakeholders pertaining to community engagement activities.

2. Describe the procedures for the collection of information.

Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

For collection instruments one (1) through 13, the respondents will only include the eight (8) BEYOND state, local and tribal government lead participants and/or the seven (7) PSP industry participants. Each of the participants has signed a memorandum of agreement (MOA) or memorandum of understanding (MOU) with the FAA which describes the data collections and frequency. Each participant is expected to submit the information as a condition of their participation in the program. The selection process for each program provided the FAA an opportunity to choose diverse groups of participants working on different types of UAS operations in a variety of environments. The only collection instruments to be submitted more frequently than quarterly include flight reports and maintenance reports. Each of these is needed by FAA on a monthly basis to inform the durability and reliability of various aircraft and provide FAA important data related to the safety and performance of UAS operations. There will be no statistical methodology for stratification and sample selection.

Regarding use of questions in the Community Engagement Tool there will also be no statistical methodology for stratification and sample selection. Estimation procedures are based on participants’ community engagement plans. Those plans show that most public engagement will be targeted to local populations or stakeholders directly affected by the operations, meaning use of Community Engagement Tool questions will be relatively limited for now as operations have been limited in number and scope, and increase yearly as operational concepts are proven and expanded to new areas and from more rural areas to suburban and urban areas over time. Most outreach will be in the form of small events such as town halls or communications directly with local neighborhoods. We estimate that on average, the questions will reach no more than 5,000 people per year per participant the first year, and perhaps double each year thereafter. That would bring the total to 35,000 members of the public potentially asked to answer the survey per participant over the course of three years, annualized to 11,667 per participant per year. There are no unusual problems that could require specialized sampling procedures.

Participants may use as many or as few of the questions provided in the community engagement tool as they choose based upon their community engagement plans and operations. Information collected is expected to be submitted semi-annually in the BEYOND program and quarterly in the PSP, in conjunction with other submissions for those programs, to minimize the potential burden on participants. We estimate that the burden on respondents will be a maximum of 10 minutes, and participation will be on a voluntary basis. In addition, all efforts will be made to ensure that the requested data is collected and submitted electronically further reducing the time and burden of the compliance.

3. Describe methods to maximize response rates.

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

For collection instruments one (1) through 13, each of the program participants has signed a memorandum of agreement (MOA) or memorandum of understanding (MOU) with the FAA which describes the data collections and frequency. Each participant is expected to submit the information as a condition of their participation in the program. Some of the information to be submitted to the FAA is program specific, and will not need to be statistically significant. For data that is of use FAA-wide for decision-making purposes, the data collected from the programs will be aggregated with other FAA UAS data collections, and therefore does not require statistical significance on its own.

In regards to the Community Engagement Tool, the program participants will be generally encouraged to conduct community engagement activities, including deploying questions from the tool or other operation-appropriate questions and collecting relevant information as much as possible, but their level of participation is at their discretion. Since data collection is not meant to be statistical in nature, nor to provide statistically significant information for policy or decision-making, no extraordinary efforts are expected to be undertaken in order to maximize response rates and/or to deal with issues of non-response.

The BEYOND and PSP programs are meant to be short-term programs with the objective of informing FAA efforts to integrate UAS into the NAS. As necessary, future programs with their own clearance(s) may be needed to further these goals. If so, data, best practices and lessons learned collected from BEYOND and PSP programs will be used to inform those future data collection efforts. However, as UAS become more fully integrated into the NAS, these data collection efforts may no longer be needed.

4. Describe tests of procedures and methods to be undertaken.

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

For collection instruments one (1) through 13, each of the program participants has signed a memorandum of agreement (MOA) or memorandum of understanding (MOU) with the FAA which describes the data collections and frequency. Participants in the programs were given opportunities to comment on and shape the information collections to ensure collection and submission is feasible. FAA lines of business were also given opportunities to review and give feedback to ensure the data collections have clear lines of sight to for FAA’s use of the data. As the data collections do not require statistical significance to be useful, we do not plan to conduct tests of procedures and methods to be undertaken.

For the Community Engagement Tool, since data collection completely optional, is not meant to be statistical in nature, nor to provide statistically significant information for policy or decision-making, we do not plan to conduct tests of procedures and methods to be undertaken.

5. Provide the names of consultants and the person who will collect and analyze the information.

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

No individuals were consulted on statistical aspects of the data collections because they are not meant to be statistical in nature nor to provide statistically significant information. Additionally, the Community Engagement Tool is not intended for policy or decision-making, merely to assist the participants with their community engagement activities and potentially be used in the development of best practices and lessons learned.

Members of the BEYOND and PSP programs within the FAA’s Unmanned Aircraft Systems (UAS) organization will be tasked with analyzing the information for the agency and disseminating results to internal and external stakeholders as appropriate. Current project managers for the programs:

Kim Merchant

Tia Cantrell

Joshua Eastlee

Jennifer Audette

Autumn Alderdice

Mack Martinez

Rogan Flowers

Thomas Beatty

John Sawyer

Samuel Colasanti

The Acting Deputy Director of the BEYOND program, Corbin T. Jones, serves as the primary point of contact. Mr. Jones’ telephone number is (202) 267-3283.