doc. Mgr. Matěj Hoffmann, Ph.D.

As robots are leaving safety fences and starting to share workspaces and even living spaces with humans, they need to dynamically adapt to unpredictable interactions with people and guarantee safety at every moment. On the rapidly growing market for collaborative robots, safety is ensured through specific technologies such as force limitations by design or contact detection and stopping relying on force measurements. Humans, however, possess awareness of their body in space drawing on dynamic, context-dependent fusion of multimodal sensory information, which makes them adaptive, flexible, and versatile. We will add important new dimensions to physical human-robot interaction: (1) we will use artificial electronic skins, developed only recently; (2) we will extend the skin space to the space around it, giving rise to a protective safety margin following the body parts in space - inspired by the peripersonal space in humans; (3) due to the multimodal nature of our approach, redundant information about the robot itself and its environment is typically available, which facilitates continuous self-calibration as well as reasoning with uncertainty. We're collaborating with leading research labs in Europe as well as industrial partners (e.g., KUKA Corporate Research, Airskin). https://sites.google.com/site/matejhof/research/cobots-and-hri https://sites.google.com/site/matejhof/research/touch-and-selfcalibration http://cmp.felk.cvut.cz/projects/body-schema

How do babies learn about their bodies? Newborns probably do not have a holistic perception of their body; instead they are starting to pick up correlations in the streams of individual sensory modalities (in particular visual, tactile, proprioceptive). The structure in these streams allows them to learn the first models of their bodies in space. The mechanisms behind these processes are largely unclear. In collaboration with developmental and cognitive psychologists and neuroscientists (New Orleans, Paris, Bielefeld), we want to shed more light on this topic: through implementations in humanoid robots, the goal is to develop concrete, embodied models of the development of multimodal (tactileproprioceptive-visual) representations of the body and the space surrounding it. Access to humanoid robots with wholebody artificial sensitive skin provides a key enabling technology. At the same time, the algorithms developed pave the way for robots with "whole-body awareness" and find applications in automatic robot calibration and safe physical human-robot interaction.

This topic investigates neuromorphic sensing and computing, focusing on event-based vision and spiking neural networks (SNNs) for low-power, low-latency robotic applications. Vision is a fundamental modality for both animals and robots, essential for perceiving and interacting with the environment. However, conventional vision systems rely on high-rate frame acquisition and continuous processing, making them computationally intensive and energy demanding, particularly in active vision scenarios that require constant visual input. Event-based sensing provides a biologically inspired alternative by capturing only changes in the scene. The resulting sparse and asynchronous output significantly reduces latency, energy consumption, and data redundancy, and has demonstrated strong potential in real-time robotic applications. When combined with neuromorphic computing (non-von Neumann architectures) that emulate brain-like processing through SNNs, these architectures enable highly efficient systems for perception and processing. The objective of this research is to develop and deploy novel algorithms on robotic platforms such as the humanoid robot iCub. Specific areas of investigation include active vision, gain modulation, visual odometry, motion detection, visual attention, stereopsis, event-driven sensing in modalities such as touch, and biologically inspired models of perception. By integrating event-based sensing with neuromorphic processing, this work aims to advance robotic vision systems that are both adaptive and energy efficient. The resulting algorithms will contribute to the development of next-generation autonomous robots capable of operating in real-time within complex, dynamic environments.

The goal is to use different robotic arms and grippers to explore various objects and collect data from proprioceptive, tactile, and force feedback. Different clustering or classification algorithms will be employed on this data to differentiate between the objects, focusing in particular on properties that can only be extracted from haptic exploration (manipulating the objects) such as stiffness, elasticity, etc. In a second step, the choice of manual actions that aid recognition will be studied. Finally, priors extracted from vision or other sources (Internet – linguistic description) as well as visual feedback (normal or RGB-D cameras) can be also employed. The main research questions are as follows: 1) What are the optimal exploratory actions for a given object, gripper and the object properties we want to learn about? 2) How are the object properties that can be learned dependent on (i) the gripper used, (ii) the exploratory action (sequence)?

Endowing robots with touch perception has been recognized as a challenging task for several decades. Tactile sensing for manipulation has received the most attention (e.g., Lambeta et al. 2020), enabling haptic or visuo-haptic object exploration (Navarro-Guerrero et al. 2023). Whole-body artificial skins have been receiving increasing attention recently. Despite a large number of technologies having been developed to date (see e.g. Bartolozzi et al. 2016 for a survey), the tactile perception capabilities of artificial tactile sensors and electronic skins are still limited. This Ph.D. thesis will explore new application areas of robot tactile sensing in haptic exploration, physical human-robot interaction for safety (e.g., Švarný et al. 2022) and for communication in social robotics. The research will involve new sensing and mechatronic solutions through collaboration with the Institute of Physics of the Czech Academy of Sciences (https://www.fzu.cz) using advanced tactile sensor materials and technologies (Kumar et al. 2022). Next to tactile sensing, proximity sensing exploiting capacitance or whisker-like designs may be explored. Finally, new biologically inspired ways of processing tactile data (Liu et al. 2022) can be investigated.

As robots are leaving safety fences and starting to share workspaces and even living spaces with humans, they need to dynamically adapt to unpredictable interactions with people and guarantee safety at every moment. Motion planning for static environments and open-loop trajectory execution is insufficient in these contexts. This topic explores algorithms for robot perception and control for interaction in dynamic, including human-populated, environments. We specifically leverage sensitive skins covering large areas of the robot bodies. Tactile sensors alone are used for contact detection and corresponding avoidance. In combination with vision, they endow the robot with “whole-body awareness” - personal and peripersonal space representations inspired by humans. We will develop control algorithms drawing on multimodal inputs and deploy them in robot manipulation and human-robot collaboration scenarios.