Publications
Applicability and surrogacy of uncorrelated airspace encounter models at low altitudes
July 1, 2021
Journal Article
Published in:
J. Air Transport., Vol. 29, No. 3, July-September 2021, pp. 137-41.
Topic:
R&D area:
R&D group:
Summary
National Airspace System (NAS) is a complex and evolving system that enables safe and efficient aviation. Advanced air mobility concepts and new airspace entrants, such as unmanned aircraft, must integrate into the NAS without degrading overall safety or efficiency. For instance, regulations, standards, and systems are required to mitigate the risk of a midair collision between aircraft. Monte Carlo simulations have been a foundational capability for decades to develop, assess, and certify aircraft conflict avoidance systems. These are often validated through human-in-the-loop experiments and flight testing. For example, an update to the Traffic Collision Avoidance System (TCAS) mandated for manned aircraft was validated in part using this approach [1]. For many aviation safety studies, manned aircraft behavior is represented using the MIT Lincoln Laboratory statistical encounter models [2–5]. The original models [2–4] were developed from 2008 to 2013 to support safety simulations for altitudes above 500 feet above ground level (AGL). However, these models were not sufficient to assess the safety of smaller unmanned aerial systems (UAS) operations below 500 feet AGL and fully support the ASTM F38 and RTCA SC-147 standards efforts. In response, newer models [5–7] with altitude floors below 500 feet AGL have been in development since 2018. Many of the models assume that aircraft behavior is uncorrelated and not dependent on air traffic services or nearby aircraft. The models were trained using observations of cooperative aircraft equipped with transponders, but data sources and assumptions vary. The newer models are organized by aircraft types of fixed-wing multi-engine, fixed-wing single engine, and rotorcraft, whereas the original models do not consider aircraft type. Our research objective was to compare the various uncorrelated models of conventional aircraft and identify how the models differ. Particularly if models of rotorcraft were sufficiently different from models of fixed-wing aircraft to require type-specific models. The scope of this work was limited to altitudes below 5000 feet AGL, the expected altitude ceiling for many new airspace entrants. The scope was also informed by the Federal Aviation Administration (FAA) UAS Integration Office and Alliance for System Safety of UAS through Research Excellence (ASSURE). The primary contribution is guidance on which uncorrelated models to leverage when evaluating the performance of a collision avoidance system designed for low-altitude operations, such as prescribed by the ASTM F3442 detect and avoid standard for smaller UAS [8]. We also address which models can be surrogates for non-cooperative aircraft without transponders. All models and software used are publicly available under open source licenses [9].
Summary
National Airspace System (NAS) is a complex and evolving system that enables safe and efficient aviation. Advanced air mobility concepts and new airspace entrants, such as unmanned aircraft, must integrate into the NAS without degrading overall safety or efficiency. For instance, regulations, standards, and systems are required to mitigate the...
READ MORE
The need for spectrum and the impact on weather observations
July 1, 2021
Journal Article
Published in:
Bull. Am. Meteorol. Soc., Vol. 102, No. 7, July 2021, E1402-7.
Topic:
Summary
One of the most significant challenges—and potential opportunities—for the scientific community is society's insatiable need for the radio spectrum. Wireless communication systems have profoundly impacted the world's economies and its inhabitants. Newer technological uses in telemedicine, Internet of Things, streaming services, intelligent transportation, etc., are driving the rapid development of 5G/6G (and beyond) wireless systems that demand ever-increasing bandwidth and performance. Without question, these wireless technologies provide an important benefit to society with the potential to mitigate the economic divide across the world. Fundamental science drives the development of future technologies and benefits society through an improved understanding of the world in which we live. Often, these studies require use of the radio spectrum, which can lead to an adversarial relationship between ever evolving technology commercialization and the quest for scientific understanding. Nowhere is this contention more acute than with atmospheric remote sensing and associated weather forecasts (Saltikoff et al. 2016; Witze 2019), which was the theme for the virtual Workshop on Spectrum Challenges and Opportunities for Weather Observations held in November 2020 and hosted by the University of Oklahoma. The workshop focused on spectrum challenges for remote sensing observations of the atmosphere, including active (e.g., weather radars, cloud radars) and passive (e.g., microwave imagers, radiometers) systems for both spaceborne and ground-based applications. These systems produce data that are crucial for weather forecasting—we chose to primarily limit the workshop scope to forecasts up to 14 days, although some observations (e.g., satellite) cover a broader range of temporal scales. Nearly 70 participants from the United States, Europe, South America, and Asia took part in a concentrated and intense discussion focused not only on current radio frequency interference (RFI) issues, but potential cooperative uses of the spectrum ("spectrum sharing"). Equally important to the workshop's international makeup, participants also represented different sectors of the community, including academia, industry, and government organizations. Given the importance of spectrum challenges to the future of scientific endeavor, the U.S. National Science Foundation (NSF) recently began the Spectrum Innovation Initiative (SII) program, which has a goal to synergistically grow 5G/6G technologies with crucial scientific needs for spectrum as an integral part of the design process. The SII program will accomplish this goal in part through establishing the first nationwide institute focused on 5G/6G technologies and science. The University of California, San Diego (UCSD), is leading an effort to compete for NSF SII funding to establish the National Center for Wireless Spectrum Research. As key partners in this effort, the University of Oklahoma (OU) and The Pennsylvania State University (PSU) hosted this workshop to bring together intellectual leaders with a focus on impacts of the spectrum revolution on weather observations and numerical weather prediction.
Summary
One of the most significant challenges—and potential opportunities—for the scientific community is society's insatiable need for the radio spectrum. Wireless communication systems have profoundly impacted the world's economies and its inhabitants. Newer technological uses in telemedicine, Internet of Things, streaming services, intelligent transportation, etc., are driving the rapid development of...
READ MORE
Towards the next generation operational meteorological radar
July 1, 2021
Journal Article
Published in:
Bull. Am. Meteorol. Soc., Vol. 102, No. 7, July 2021, pp. E1357-82.
Topic:
R&D area:
Summary
This article summarizes research and risk reduction that will inform acquisition decisions regarding NOAA's future national operational weather radar network. A key alternative being evaluated is polarimetric phased-array radar (PAR). Research indicates PAR can plausibly achieve fast, adaptive volumetric scanning, with associated benefits for severe-weather warning performance. We assess these benefits using storm observations and analyses, observing system simulation experiments, and real radar-data assimilation studies. Changes in the number and/or locations of radars in the future network could improve coverage at low altitude. Analysis of benefits that might be so realized indicates the possibility for additional improvement in severe weather and flash-flood warning performance, with associated reduction in casualties. Simulations are used to evaluate techniques for rapid volumetric scanning and assess data quality characteristics of PAR. Finally, we describe progress in developing methods to compensate for polarimetric variable estimate biases introduced by electronic beam-steering. A research-to-operations (R2O) strategy for the PAR alternative for the WSR-88D replacement network is presented.
Summary
This article summarizes research and risk reduction that will inform acquisition decisions regarding NOAA's future national operational weather radar network. A key alternative being evaluated is polarimetric phased-array radar (PAR). Research indicates PAR can plausibly achieve fast, adaptive volumetric scanning, with associated benefits for severe-weather warning performance. We assess these...
READ MORE
Mobile capabilities for micro-meteorological predictions: FY20 Homeland Protection and Air Traffic Control Technical Investment Program
March 26, 2021
Project Report
Published in:
MIT Lincoln Laboratory Report TIP-146
Topic:
R&D area:
Summary
Existing operational numerical weather forecast systems are geographically too coarse and not sufficiently accurate to adequately support future needs in applications such as Advanced Air Mobility, Unmanned Aerial Systems, and wildfire forecasting. This is especially true with respect to wind forecasts. Principal factors contributing to this are the lack of observation data within the atmospheric boundary layer and numerical forecast models that operate on low-resolution grids. This project endeavored to address both of these issues. Firstly, by development and demonstration of specially equipped fixed-wing drones to collect atmospheric data within the boundary layer, and secondly by creating a high-resolution weather research forecast model executing on the Lincoln Laboratory Supercomputing Center. Some success was achieved in the development and flight testing of the specialized drones. Significant success was achieved in the development of the high-resolution forecasting system and demonstrating the feasibility of ingesting atmospheric observations from small airborne platforms.
Summary
Existing operational numerical weather forecast systems are geographically too coarse and not sufficiently accurate to adequately support future needs in applications such as Advanced Air Mobility, Unmanned Aerial Systems, and wildfire forecasting. This is especially true with respect to wind forecasts. Principal factors contributing to this are the lack of...
READ MORE
Advanced Air Mobility assessment framework: FY20 Homeland Protection and Air Traffic Control Technical Investment Program
March 24, 2021
Project Report
Published in:
MIT Lincoln Laboratory Report TIP-145
Summary
Advanced Air Mobility encompasses emerging aviation technologies that transport people and cargo between local, regional, or urban locations that are currently underserved by aviation and other transportation modalities. The disruptive nature of these technologies has pushed industry, academia, and governments to devote significant investments to understand their impact on airspace risk, operational procedures, and passengers. A flexible framework was designed to assess the operational viability of these technologies and the sensitivity to a variety of assumptions. This framework is used to simulate an initial AAM implementation scenario in New York City. This scenario was created by replacing a portion of NYC taxi requests with electric vertical takeoff and landing vehicles. The framework was used to assess the sensitivity of this scenario to a variety of system assumption.
Summary
Advanced Air Mobility encompasses emerging aviation technologies that transport people and cargo between local, regional, or urban locations that are currently underserved by aviation and other transportation modalities. The disruptive nature of these technologies has pushed industry, academia, and governments to devote significant investments to understand their impact on airspace...
READ MORE
NASA Airspace Integration Detect and Avoid Phase 2: Safety Risk Management Simulation Plan
December 18, 2020
Project Report
Published in:
MIT Lincoln Laboratory Report
Topic:
R&D area:
R&D group:
Summary
RTCA has been developing Minimum Operational Performance Standards (MOPS) for Detect and Avoid (DAA) and Command and Control (C2) systems as part of Special Committee – 228 (SC-228). The Phase 1 MOPS were published in 2017 and a Phase 2 effort to revise and extend the Phase 1 MOPS is ongoing. In order for the MOPS to be fully utilized, they must be evaluated by the FAA employing the FAA's Safety Risk Management (SRM) process. In order to support the SRM process, there is a need for simulation data focused on the safety of DAA encounters. This analysis focuses on gathering information to validate the use of SC-228 MOPS compliant DAA and C2 systems to enable routine UAS operations in the National Airspace System (NAS) without a chase aircraft or visual observers. The scope of this effort aligns with the SC-228 Terms of Reference (TOR) that can be generally characterized as UAS flying IFR and receiving ATC separation services. This analysis evaluates the system safety in mixed classes B, C, D, E, and G airspaces, and includes IFR, VFR, Cooperative, and Non-Cooperative aircraft. This analysis plan describes the four analysis tasks (Section 2) and the simulation plan (Section 3) that will be executed to accomplish these tasks. It is expected that this analysis plan will be extended to include the analysis results and become the final deliverable.
Summary
RTCA has been developing Minimum Operational Performance Standards (MOPS) for Detect and Avoid (DAA) and Command and Control (C2) systems as part of Special Committee – 228 (SC-228). The Phase 1 MOPS were published in 2017 and a Phase 2 effort to revise and extend the Phase 1 MOPS is...
READ MORE
Adaptive stress testing: finding likely failure events with reinforcement learning
December 1, 2020
Journal Article
Published in:
J. Artif. Intell. Res., Vol. 69, 2020, pp. 1165-1201.
Summary
Finding the most likely path to a set of failure states is important to the analysis of safety critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates. As a result, safety validation is not only concerned about whether a failure can occur, but also discovering which failures are most likely to occur. This article presents adaptive stress testing (AST), a framework for finding the most likely path to a failure event in simulation. We consider a general black box setting for partially observable and continuous-valued systems operating in an environment with stochastic disturbances. We formulate the problem as a Markov decision process and use reinforcement learning to optimize it. The approach is simulation-based and does not require internal knowledge of the system, making it suitable for black-box testing of large systems. We present different formulations depending on whether the state is fully observable or partially observable. In the latter case, we present a modified Monte Carlo tree search algorithm that only requires access to the pseudorandom number generator of the simulator to overcome partial observability. We also present an extension of the framework, called differential adaptive stress testing (DAST), that can find failures that occur in one system but not in another. This type of differential analysis is useful in applications such as regression testing, where we are concerned with finding areas of relative weakness compared to a baseline. We demonstrate the effectiveness of the approach on an aircraft collision avoidance application, where a prototype aircraft collision avoidance system is stress tested to find the most likely scenarios of near mid-air collision.
Summary
Finding the most likely path to a set of failure states is important to the analysis of safety critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due...
READ MORE
A quantitatively derived NMAC analog for smaller unmanned aircraft systems based on unmitigated collision risk
November 19, 2020
Journal Article
Published in:
Preprints, 19 November 2020.
Summary
The capability to avoid other air traffic is a fundamental component of the layered conflict management system to ensure safe and efficient operations in the National Airspace System. The evaluation of systems designed to mitigate the risk of midair collisions of manned aircraft are based on large-scale modeling and simulation efforts and a quantitative volume defined as a near midair collision (NMAC). Since midair collisions are difficult to observe in simulation and are inherently rare events, basing evaluations on NMAC enables a more robust statistical analysis. However, an NMAC and its underlying assumptions for assessing close encounters with manned aircraft do not adequately consider the different characteristics of smaller UAS-only encounters. The primary contribution of this paper is to explore quantitative criteria to use when simulating two or more smaller UASs in sufficiently close proximity that a midair collision might reasonably occur and without any mitigations to reduce the likelihood of a midair collision. The criteria assumes a historically motivated upper bound for the collision likelihood and subsequently identify the smallest possible NMAC analogs. We also demonstrate the NMAC analogs can be used to support modeling and simulation activities.
Summary
The capability to avoid other air traffic is a fundamental component of the layered conflict management system to ensure safe and efficient operations in the National Airspace System. The evaluation of systems designed to mitigate the risk of midair collisions of manned aircraft are based on large-scale modeling and simulation...
READ MORE
Frequency of ADS-B equipped manned aircraft observed by the OpenSky Network
November 12, 2020
Conference Paper
Published in:
8th OpenSky Symp. 2020, Online, 12–13 November 2020.
Summary
To support integration of unmanned aerial systems into the airspace, the low altitude airspace needs to be characterized. Identifying the frequency of different aircraft types, such as rotorcraft or fixed wing single engine, given criteria such as altitude, airspace class, or quantity of seats can inform surveillance requirements, flight test campaigns, or simulation safety thresholds for detect and avoid systems. We leveraged observations of Automatic Dependent Surveillance-Broadcast (ADS-B) equipped aircraft by the OpenSky Network for this characterization.
Summary
To support integration of unmanned aerial systems into the airspace, the low altitude airspace needs to be characterized. Identifying the frequency of different aircraft types, such as rotorcraft or fixed wing single engine, given criteria such as altitude, airspace class, or quantity of seats can inform surveillance requirements, flight test...
READ MORE
Multi-Agent Systems Collaborative Teaming (MASCOT) definition process to create specifications for Multi-Agent System (MAS) development
November 2, 2020
Conference Paper
Published in:
25th Intl. Command and Control Research and Technology Symp., ICCRTS 2020, 2-5 November 2020.
Topic:
R&D area:
R&D group:
Summary
The US Army envisions heterogeneous teams of advanced machines and humans that will collaborate together to achieve a common mission goal. It is essential for commanders to quickly and effectively respond to dynamic mission environments with agile re-tasking and computerized aids for plan definition/redefinition, and to perform some tasks with bounded autonomy. Workload constraints limit an individual's ability to concurrently control many platforms, so some mission segments many need to be autonomous or to be quickly selected via a tactics playbook. Denied environments also dictate the need for machine participants in some mission segments to be autonomous (or semi-autonomous). A Multi-Agent System (MAS) provides a natural paradigm for describing a system of agents that work together in such environments. An agent can be a human or machine, but is generally a machine. Creating MAS systems and requirements has proved to be a formidable task due to mission complexities, the necessity to deal with unforeseen circumstances, and the general difficulty of defining autonomous behaviors. We define a process called Multi-Agent Systems Collaborative Teaming (MASCOT) Definition Process that starts with a Subject Matter Experts (SME), produces a set of agent specifications, and derives system requirements in sufficient detail to define a MAS that can be modeled in a test-bed, used for facilitation of a safety analysis, and produced into an actual system. The MASCOT process also enables concurrent development of an effects based ontology. We demonstrate the MASCOT process on an example case study to show the efficacy of our process.
Summary
The US Army envisions heterogeneous teams of advanced machines and humans that will collaborate together to achieve a common mission goal. It is essential for commanders to quickly and effectively respond to dynamic mission environments with agile re-tasking and computerized aids for plan definition/redefinition, and to perform some tasks with...
READ MORE