Shubhan Parag Patni, MSc.

Different robotic setups provide tactile feedback about the objects they interact with in different manners. This makes it difficult to transfer the information gained from haptic exploration to different setups and to humans as well. We introduce “touch primitives”, a set of object features for haptic shape representation which aim to reconstruct the shape of objects independent from the robot morphology. We investigate how precisely the primitives can be extracted from household objects by a commonly used gripper, on a set of objects that vary in size, shape and stiffness.

Robot morphology, which includes the physical dimension and shape but also the placement and type of actuators and sensors, is highly variable. This also applies to different robot hand and grippers, equipped with force or tactile sensors. Unlike in computer vision, where information from cameras is robot and largely camera-independent, haptic information is morphology-dependent, which makes it difficult to transfer object recognition and other pipelines between setups. In this work, we introduce a shape reconstruction and grasping task to evaluate the success of haptic information transfer between robotic setups, and propose feature descriptors that can help in standardizing the haptic representation of shapes across different robotic setups.

Due to the large variety of tactile and proprioceptive sensors available for integration with robotic grippers, the data structures for data collected on different robotic setups are different, which makes it difficult to compile and compare these datasets for robot learning. We propose “Touch Primitives”—a gripper-independent representation for the haptic exploration of object shapes which can be generalized across different gripper and sensor combinations. An exploration and grasping task is detailed to test the efficacy of the proposed touch primitive features.

We investigated the use of impact sounds generated during exploratory behaviors in a robotic manipulation setup as cues for predicting object surface material and for recognizing individual objects. We collected and make available the YCB-impact sounds dataset which includes over 3,500 impact sounds for the YCB set of everyday objects lying on a table. Impact sounds were generated in three modes: (i) human holding a gripper and hitting, scratching, or dropping the object; (ii) gripper attached to a teleoperated robot hitting the object from the top; (iii) autonomously operated robot hitting the objects from the side with two different speeds. A convolutional neural network (ResNet34) is trained from scratch to recognize the object material (steel, aluminium, hard plastic, soft plastic, other plastic, ceramic, wood, paper/cardboard, foam, glass, rubber) from a single impact sound. On the manually collected dataset with more variability in the action, nearly 60\% accuracy for the test set (unseen objects) was achieved. On a robot setup and a stereotypical poking action from top, accuracy of 85% was achieved. This performance drops to 79% if multiple exploratory actions are combined. Individual objects from the set of 75 objects can be recognized with a 79% accuracy. This work demonstrates promising results regarding the possibility of using sound for recognition in tasks like single-stream recycling where objects have to be sorted based on their material composition.