Ing. Zdeněk Rozsypálek

When a mobile robot is asked to navigate intelligently in an environment, it needs to estimate its own and the environment's state. One of the popular methods for robot state and position estimation is particle filtering (PF). Visual Teach and Repeat (VT & R) is a type of navigation that uses a camera to navigate the robot along the previously traversed path. Particle filters are usually used in VT & R to fuse data from odometry and camera to estimate the distance traveled along the path. However, in VT & R, there are other valuable states that the robot can benefit from, especially when moving through changing environments. We propose a multidimensional particle filter to estimate the robot state in VT & R navigation. Apart from the traveled distance, our particle filter estimates lateral and heading deviation from the taught path as well as the current appearance of the environment. This appearance is estimated using maps created across various environmental conditions recorded during the previous traversals. The joint state estimation is based on contrastive neural network architecture, allowing self-supervised learning. This architecture can process multiple images in parallel, alleviating the potential overhead caused by computing the particle filter over the maps simultaneously. We conducted experiments to show that the joint robot/environment state estimation improves navigation accuracy and robustness in a continual mapping setup. Unlike the other frameworks, which treat the robot position and environment appearance separately, our PF represents them as one multidimensional state, resulting in a more general uncertainty model for VT & R.

In practical work scenarios, it is often necessary to repeat specific tasks, which include navigating along a desired path. Visual teach and repeat systems are a type of autonomous navigation in which a robot repeats a previously taught path using a camera and dead reckoning. There have been many different teach and repeat methods proposed in the literature, but only a few are open-source. In this paper, we compare four recently published open-source methods and a Boston Dynamics proprietary solution embedded in a Spot robot. The intended use for each method is different, which has an impact on their strengths and weaknesses. When deciding which method to use, factors such as the environment and desired precision and speed should be taken into consideration. For example, in controlled artificial environments, which do not change significantly, navigation precision and speed are more important than robustness to environment variations. However, the appearance of unstructured natural environments varies over time, making robustness to changes a crucial property for outdoor navigation systems. This paper compares the speed, precision, reliability, robustness, and practicality of the available teach and repeat methods. We will outline their flaws and strengths, helping to choose the most suitable method for a particular utilization.

With increasing computation power, longer battery life and lower prices, mobile robots are becoming a viable option for many applications. When the application requires long-term autonomy in an uncontrolled environment, it is necessary to equip the robot with a navigation system robust to environmental changes. Visual Teach and Repeat (VT&R) is one such navigation system that is lightweight and easy to use. Similarly, as other methods rely on camera input, the performance of VT&R can be highly influenced by changes in the scene's appearance. One way to address this problem is to use machine learning or/and add redundancy to the sensory input. However, it is usually complicated to collect long-term datasets for given sensory input, which can be exploited by machine learning methods to extract knowledge about possible changes in the environment from the data. In this paper, we show that we can use a dataset not containing the environmental changes to train a model processing infrared images and improve the robustness of the VT&R framework by fusion with the classic method based on RGB images. In particular, our experiments show that the proposed training scheme and fusion method can alleviate the problems arising from adverse illumination changes. Our approach can broaden the scope of possible VT&R applications that require deployment in environments with significant illumination changes.

Nowadays, most agricultural robots rely on precise and expensive localisation, typically based on global navigation satellite systems (GNSS) and real-time kinematic (RTK) receivers. Unfortunately, the precision of GNSS localisation significantly decreases in environments where the signal paths between the receiver and the satellites are obstructed. This precision hampers deployments of these robots in, e.g., polytunnels or forests. An attractive alternative to GNSS is vision-based localisation and navigation. However, perceptual aliasing and landmark deficiency, typical for agricultural environments, cause traditional image processing techniques, such as feature matching, to fail. We propose an approach for an affordable pure vision-based navigation system which is not only robust to perceptual aliasing, but it actually exploits the repetitiveness of agricultural environments. Our system extends the classic concept of visual teach and repeat to visual teach and generalise (VTAG). Our teach and generalise method uses a deep learning-based image registration pipeline to register similar images through meaningful generalised representations obtained from different but similar areas. The proposed system uses only a low-cost uncalibrated monocular camera and the robot's wheel odometry to produce heading corrections to traverse crop rows in polytunnels safely. We evaluate this method at our test farm and at a commercial farm on three different robotic platforms where an operator teaches only a single crop row. With all platforms, the method successfully navigates the majority of rows with most interventions required at the end of the rows, where the camera no longer has a view of any repeating landmarks such as poles, crop row tables or rows which have visually different features to that of the taught row. For one robot which was taught one row 25 m long our approach autonomously navigated the robot a total distance of over 3.5 km, reaching a teach-generalisation gain of 140.

Visual teach and repeat navigation (VT&R) is popular in robotics thanks to its simplicity and versatility. It enables mobile robots equipped with a camera to traverse learned paths without the need to create globally consistent metric maps. Although teach and repeat frameworks have been reported to be relatively robust to changing environments, they still struggle with day-to-night and seasonal changes. This paper aims to find the horizontal displacement between prerecorded and currently perceived images required to steer a robot towards the previously traversed path. We employ a fully convolutional neural network to obtain dense representations of the images that are robust to changes in the environment and variations in illumination. The proposed model achieves state-of-the-art performance on multiple datasets with seasonal and day/night variations. In addition, our experiments show that it is possible to use the model to generate additional training examples that can be used to further improve the original model's robustness. We also conducted a real-world experiment on a mobile robot to demonstrate the suitability of our method for VT&R.

The performance of deep neural networks and the low costs of computational hardware has made computer vision a popular choice in many robotic systems. An attractive feature of deep-learned methods is their ability to cope with appearance changes caused by day-night cycles and seasonal variations. However, deep learning of neural networks typically relies on large numbers of hand-annotated images, which requires significant effort for data collection and annotation. We present a method that allows autonomous, self-supervised training of a neural network in visual teach-and-repeat (VT&R) tasks, where a mobile robot has to traverse a previously taught path repeatedly. Our method is based on a fusion of two image registration schemes: one based on a Siamese neural network and another on point-feature matching. As the robot traverses the taught paths, it uses the results of feature-based matching to train the neural network, which, in turn, provides coarse registration estimates to the feature matcher. We show that as the neural network gets trained, the accuracy and robustness of the navigation increases, making the robot capable of dealing with significant changes in the environment. This method can significantly reduce the data annotation efforts when designing new robotic systems or introducing robots into new environments. Moreover, the method provides annotated datasets that can be deployed in other navigation systems. To promote the reproducibility of the research presented herein, we provide our datasets, codes and trained models online.

Visual teach and repeat navigation (VT&R) is a framework that enables mobile robots to traverse previously learned paths. In principle, it relies on computer vision techniques that can compare the camera's current view to a model based on the images captured during the teaching phase. However, these techniques are usually not robust enough when significant changes occur in the environment between the teach and repeat phases. In this paper, we show that contrastive learning methods can learn how the environment changes and improve the robustness of a VT&R framework. We apply a fully convolutional Siamese network to register the images of the teaching and repeat phases. Their horizontal displacement between the images is then used in a visual servoing manner to keep the robot on the intended trajectory. The experiments performed on several datasets containing seasonal variations indicate that our method outperforms state-of-the-art algorithms tailored to the purpose of registering images captured in different seasons.

In this paper we present an autonomous robotic system for picking, transporting, and precisely placing magnetically graspable objects. Such a system would be especially beneficial for construction tasks where human presence is not possible, e.g. due to chemical or radioactive pollution. The system comprises of two primary components – a wheeled, mobile platform and a manipulator arm. Both are interconnected through an onboard computer and utilize various onboard sensors for estimating the state of the robot and its surroundings. By using efficient processing algorithms, data from the onboard sensors can be used in a feedback loop during all critical operational sections, resulting in a robust system capable of operating on uneven terrain and in environments without access to satellite navigation. System functionality has been proven in Challenge II of the MBZIRC 2020 competition. The Challenge required a ground robot to build an L-shaped structure of colored bricks laid in a predefined pattern. Such a mission incorporates several demanding subchallenges, spanning multiple branches of computer science, cybernetics, and robotics. Moreover, all the subchallenges had to be performed flawlessly in rapid succession, in order to complete the Challenge successfully. The extreme difficulty of the task was highlighted in the MBZIRC 2020 finals, where our system was among the only two competitors (out of 32) that managed to complete the task in autonomous mode.

Mobile manipulators have the potential to revolutionize modern agriculture, logistics and manufacturing. In this work, we present the design of a ground-based mobile manipulator for automated structure assembly. The proposed system is capable of autonomous localization, grasping, transportation and deployment of construction material in a semi-structured environment. Special effort was put into making the system invariant to lighting changes, and not reliant on external positioning systems. Therefore, the presented system is self-contained and capable of operating in outdoor and indoor conditions alike. Finally, we present means to extend the perceptive radius of the vehicle by using it in cooperation with an autonomous drone, which provides aerial reconnaissance. Performance of the proposed system has been evaluated in a series of experiments conducted in real-world conditions.