Ing. Daniel Heřt

This paper presents a system for autonomous cooperative wall building with a team of Unmanned Aerial Vehicles (UAVs). The system was developed for Challenge 2 of the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020. The wall-building scenario of Challenge 2 featured an initial stack of bricks and wall structure where the individual bricks had to be placed by a team of three UAVs. The objective of the task was to maximize collected points for placing the bricks within the restricted construction time while following the prescribed wall pattern. The proposed approach uses initial scanning to find a priori unknown locations of the bricks and the wall structure. Each UAV is then assigned to individual bricks and wall placing locations and further performs grasping and placement using onboard resources only. The developed system consists of methods for scanning a given area, RGB-D detection of bricks and wall placement locations, precise grasping and placing of bricks, and coordination of multiple UAVs. The paper describes the overall system, individual components, experimental verification in demanding outdoor conditions, the achieved results in the competition, and lessons learned. The presented CTU-UPenn-NYU approach achieved the overall best performance among all participants to win the MBZIRC competition by collecting the highest number of points by correct placement of a high number of bricks.

In this paper, we introduce a distributed autonomous flocking behavior of Unmanned Aerial Vehicles (UAVs) in demanding outdoor conditions, motivated by search and rescue applications. We propose a novel approach for decentralized swarm navigation in the direction of a candidate object of interest (OOI) based on real-time detections from onboard RGB cameras. A novel self-adaptive communication strategy secures an efficient change of swarm azimuth to a higher priority direction based on the real-time detections. We introduce a local visual communication channel that establishes a network connection between neighboring robots without explicit communication to achieve high reliability and scalability of the system. As a case study, this novel method is applied for the deployment of a UAV swarm towards detected OOI for closer inspection and verification. The results of simulations and real-world experiments have verified the intended behavior of the swarm system for the detection of true positive and false positive OOI, as well as for cooperative environment exploration.

This paper presents a modular autonomous Unmanned Aerial Vehicle (UAV) platform called the Multi-robot System (MRS) Drone that can be used in a large range of indoor and outdoor applications. The MRS Drone features unique modularity changes in actuators, frames, and sensory configuration. As the name suggests, the platform is specially tailored for deployment within a MRS group. The MRS Drone contributes to the state-of-the-art of UAV platforms by allowing smooth real-world deployment of multiple aerial robots, as well as by outperforming other platforms with its modularity. For real-world multi-robot deployment in various applications, the platform is easy to both assemble and modify. Moreover, it is accompanied by a realistic simulator to enable safe pre-flight testing and a smooth transition to complex real-world experiments. In this manuscript, we present mechanical and electrical designs, software architecture, and technical specifications to build a fully autonomous multi UAV system. Finally, we demonstrate the full capabilities and the unique modularity of the MRS Drone in various real-world applications that required a diverse range of platform configurations.

This paper presents a novel approach for autonomous cooperating UAVs in search and rescue operations in subterranean domains with complex topology. The proposed system was ranked second in the Virtual Track of the DARPA SubT Finals as part of the team CTU-CRAS-NORLAB. In contrast to the winning solution that was developed specifically for the Virtual Track, the proposed solution also proved to be a robust system for deployment onboard physical UAVs flying in the extremely harsh and confined environment of the real-world competition. The proposed approach enables fully autonomous and decentralized deployment of a UAV team with seamless simulation-to-world transfer, and proves its advantage over less mobile UGV teams in the flyable space of diverse environments. The main contributions of the paper are present in the mapping and navigation pipelines. The mapping approach employs novel map representations — SphereMap for efficient risk-aware long-distance planning, FacetMap for surface coverage, and the compressed topological-volumetric LTVMap for allowing multi-robot cooperation under low-bandwidth communication. These representations are used in navigation together with novel methods for visibility-constrained informed search in a general 3D environment with no assumptions about the environment structure, while balancing deep exploration with sensor-coverage exploitation. The proposed solution also includes a visual-perception pipeline for on-board detection and localization of objects of interest in four RGB stream at 5 Hz each without a dedicated GPU. Apart from participation in the DARPA SubT, the performance of the UAV system is supported by extensive experimental verification in diverse environments with both qualitative and quantitative evaluation.

The Mohamed Bin Zayed International Robotics Challenge (MBZIRC) is a prestigious, biennial competition aimed at furthering the state of the art in the field of autonomous robotics. In this paper, we present our solution to one of the tasks in the MBZIRC 2020 competition, which design won second place in Challenge 1 and first place in the Grand Challenge of the competition. This paper focuses specifically on the popping task of multiple balloons by multiple Micro Aerial Vehicles (MAVs). This task required a rapid and robust performance to compete with systems from other expert robotic teams from around the world. In this task, a team of autonomous MAV’s had to seek and attack several balloons positioned throughout the competition arena. The novel fast autonomous searching for multiple targets in 3D, their reliable detection, precise relative state estimation, and agile motion planning algorithms are presented in this paper, together with an application for general tasks of 3D target capturing. With a primary focus on reliability, the methods reported in this paper and the entire, complex multi-agent system were successfully verified in both extreme conditions of the desert and the MBZIRC competition. An evaluation of the proposed methods using data from the competition and additional separate datasets is presented. The relevant code of our implementation has been made publicly available for the robotics community.

In this paper, a novel approach for autonomously catching fast flying objects is presented, as inspired by the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020. In this competition, an autonomous Unmanned Aerial Vehicle (UAV) was used to intercept a ball carried by a fast flying drone. The presented solution utilizes a 3D LiDAR sensor for quick and robust target detection. The trajectory of the target is estimated and predicted to select a suitable interception position. The interceptor UAV is navigated into the interception position to safely approach the target. The interception position is frequently being adjusted based on the updated estimation and prediction of the target’s motion to ensure that the ball is caught in the dedicated onboard net. After a successful interception is detected, the UAV lands in a designated landing area. The proposed concept was intensively tested and refined in demanding outdoor conditions with strong winds and varying perception conditions to achieve the robustness required by both the demanding application and the competition. In the MBZIRC 2020 competition, our solution scored second place in Challenge 1 and first place in a combined Grand Challenge. This manuscript will provide a detailed description of the applied methods and an evaluation of our approach with data collected from real-world experiments. In addition, we present achievements of our R&D towards the transition from the MBZIRC competition to an autonomous drone interceptor, which was the main motivation of this challenge.

This paper presents a family of autonomous Unmanned Aerial Vehicles (UAVs) platforms designed for a diverse range of indoor and outdoor applications. The proposed UAV design is highly modular in terms of used actuators, sensor configurations, and even UAV frames. This allows to achieve, with minimal effort, a proper experimental setup for single, as well as, multi-robot scenarios. Presented platforms are intended to facilitate the transition from simulations, and simplified laboratory experiments, into the deployment of aerial robots into uncertain and hard-to-model real-world conditions. We present mechanical designs, electric configurations, and dynamic models of the UAVs, followed by numerous recommendations and technical details required for building such a fully autonomous UAV system for experimental verification of scientific achievements. To show strength and high variability of the proposed system, we present results of tens of completely different real-robot experiments in various environments using distinct actuator and sensory configurations.

The problem of safe interception of multiple intruder UAVs by a team of cooperating autonomous aerial vehicles is addressed in this paper. The presented work is motivated by the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020 where this task was simplified to an interaction with a set of static and dynamic objects (balloons and a UAV), and by a real autonomous aerial interception system of Eagle.One that our team has been working on. We propose a general control, perception, and coordination system for the fast and reliable interception of targets in a 3D environment relying only on onboard sensors and processing. The proposed methods and the entire complex multi-robot sys- tem were successfully verified in demanding desert conditions, with the main focus on reliability and fast deployment. In the MBZIRC competition, the proposed approach exhibited the greatest reliability and fastest solution. It was crucial to our team in winning the entire competition and achieving the second place in the intruder UAV interception scenario.

We present a multirotor Unmanned Aerial Vehicle (UAV) control and estimation system for supporting replicable research through realistic simulations and real-world experiments. We propose a unique multi-frame localization paradigm for estimating the states of a UAV in various frames of reference using multiple sensors simultaneously. The system enables complex missions in GNSS and GNSS-denied environments, including outdoor-indoor transitions and the execution of redundant estimators for backing up unreliable localization sources. Two feedback control designs are presented: one for precise and aggressive maneuvers, and the other for stable and smooth flight with a noisy state estimate. The proposed control and estimation pipeline are constructed without using the Euler/Tait-Bryan angle representation of orientation in 3D. Instead, we rely on rotation matrices and a novel heading-based convention to represent the one free rotational degree-of-freedom in 3D of a standard multirotor helicopter. We provide an actively maintained and well-documented open-source implementation, including realistic simulation of UAV, sensors, and localization systems. The proposed system is the product of years of applied research on multi-robot systems, aerial swarms, aerial manipulation, motion planning, and remote sensing. All our results have been supported by real-world system deployment that subsequently shaped the system into the form presented here. In addition, the system was utilized during the participation of our team from the Czech Technical University in Prague in the prestigious MBZIRC 2017 and 2020 robotics competitions, and also in the DARPA Subterranean challenge. Each time, our team was able to secure top places among the best competitors from all over the world.

In this letter we present an autonomous system intended for aerial monitoring, inspection and assistance in Search and Rescue (SAR) operations within a constrained workspace. The proposed system is designed for deployment in demanding real-world environments with extremely narrow passages only slightly wider than the aerial platform, and with limited visibility due to the absence of illumination and the presence of dust. The focus is on precise localization in an unknown environment, high robustness, safety and fast deployment without any need to install an external infrastructure such as an external computer and localization system. These are the main requirements of the targeted SAR scenarios. The performance of the proposed system was successfully evaluated in the Tunnel Circuit of the DARPA Subterranean Challenge, where the UAV cooperated with ground robots to precisely localize artifacts in a coal mine tunnel system. The challenge was unique due to the intention of the organizers to emulate the unpredictable conditions of a real SAR operation, in which there is no prior knowledge of the obstacles that will be encountered.

The Subterranean Challenge (SubT) is a contest organised by the Defense Advanced Research Projects Agency (DARPA). The contest reflects the requirement of increasing safety and efficiency of underground search-and-rescue missions. In the SubT challenge, teams of mobile robots have to detect, localise and report positions of specific objects in an underground environment. This paper provides a description of the multi-robot heterogeneous exploration system of our CTU-CRAS team, which scored third place in the Tunnel Circuit round, surpassing the performance of all other non-DARPA-funded competitors. In addition to the description of the platforms, algorithms and strategies used, we also discuss the lessons-learned by participating at such contest.

This paper presents a novel approach for control and motion planning of formations of multiple unmanned micro aerial vehicles(multi-rotor helicopters, in the literature also often called unmanned aerial vehicles—UAVs or unmanned aerial system—UAS) in cluttered GPS-denied on straitened environments. The proposed method enables us to autonomously design complexmaneuvers of a compact Micro Aerial Vehicles (MAV) team in a virtual-leader-follower scheme. The results of the motionplanning approach and the required stability of the formation are achieved by migrating the virtual leader along with the hullsurrounding the formation. This enables us to suddenly change the formation motion in all directions, independently from thecurrent orientation of the formation, and therefore to fully exploit the maneuverability of small multi-rotor helicopters. Theproposed method was verified and its performance has been statistically evaluated in numerous simulations and experimentswith a fleet of MAVs.

This paper addresses the perception, control, and trajectory planning for an aerial platform to identify and land on a moving car at 15 km/hr. The hexacopter unmanned aerial vehicle (UAV), equipped with onboard sensors and a computer, detects the car using a monocular camera and predicts the car future movement using a nonlinear motion model. While following the car, the UAV lands on its roof, and it attaches itself using magnetic legs. The proposed system is fully autonomous from takeoff to landing. Numerous field tests were conducted throughout the year‐long development and preparations for the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2017 competition, for which the system was designed. We propose a novel control system in which a model predictive controller is used in real time to generate a reference trajectory for the UAV, which are then tracked by the nonlinear feedback controller. This combination allows to track predictions of the car motion with minimal position error. The evaluation presents three successful autonomous landings during the MBZIRC 2017, where our system achieved the fastest landing among all competing teams.

In this letter, a novel approach to fast three-dimensional (3-D) localization of flying objects for their interception by a micro aerial vehicle (MAV) is presented. The proposed method utilizes a depth image from a stereo camera to facilitate onboard detection of drones, flying in its proximity. The method does not rely on using any kind of markers, which enables localization of non-cooperating drones. This approach strongly relaxes the requirements on the drones to be detected, and the detection algorithm is computationally undemanding enough to process images online, onboard an MAV with limited computational resources. This allows using the detection system in the control feedback of an autonomous aerial intercepting system. Output of the detection algorithm is filtered by a 3-D multi-target tracking algorithm to reduce false positives, preserve temporal consistency of the detections, and to predict positions of the drones (e.g., to compensate camera and processing delays). We demonstrate the importance of the advances in flying object localization, presented in this letter, in an experiment with an intruder-interceptor scenario, which would be unfeasible using state-of-the-art detection and localization methods.

Autonomous Micro Aerial Vehicles have the potential to assist in real life tasks involving grasping and transportation, but not before solving several difficult research challenges. In this work, we address the design, control, estimation, and planning problems for cooperative localization, grasping, and transportation of objects in challenging outdoor scenarios. We demonstrate an autonomous team of MAVs able to plan safe trajectories for manipulation of ferrous objects, while guaranteeing inter-robot collision avoidance and automatically creating a map of the objects in the environment. Our solution is predominantly distributed, allowing the team to pick and transport ferrous disks to a final destination without collisions. This result is achieved using a new magnetic gripper with a novel feedback approach, enabling the detection of successful grasping. The gripper design and all the components to build a platform are clearly provided as open-source hardware for reuse by the community. Finally, the proposed solution is validated through experimental results where difficulties include inconsistent wind, uneven terrain, and sandy conditions.

We propose a novel approach for optimal trajectory tracking for unmanned aerial vehicles (UAV), using a linear model predictive controller (MPC) in combination with non-linear state feedback. The solution relies on fast onboard simulation of the translational dynamics of the UAV, which is guided by a linear MPC. By sampling the states of the virtual UAV, we create a control command for fast non-linear feedback, which is capable of performing agile maneuvers with high precision. In addition, the proposed pipeline provides an interface for a decentralized collision avoidance system for multi-UAV scenarios. Our solution makes use of the long prediction horizon of the linear MPC and allows safe outdoors execution of multi-UAV experiments without the need for in-advance collision-free planning. The practicality of the tracking mechanism is shown in combination with priority-based collision resolution strategy, which performs sufficiently in experiments with up to 5 UAVs. We present a statistical and experimental evaluation of the platform in both simulation and real-world examples, demonstrating the usability of the approach.

A novel approach for control and motion planning of formations of multiple unmanned micro aerial vehicles (MAVs), also referred to as unmanned aerial vehicles (UAVs) - multirotor helicopters, in cluttered GPS-denied environments is presented in this paper. The proposed method enables autonomously to design complex maneuvers of a compact MAV team in a virtual-leaderfollower scheme. The feasibility of obtained results of the motion planning approach and the required stability of the formation is achieved by migrating the virtual leader along a hull surrounding the formation. This enables us to suddenly change formation motion in all directions, independently of actual orientation of the formation.