Ing. Michal Neoral

We propose a method for discovery and segmentation of objects that are, or their parts are, independently moving in the scene. Given three monocular video frames, the method outputs semantically meaningful regions, i.e. regions corresponding to the whole object, even when only a part of it moves. The architecture of the CNN-based end-to-end method, called Raptor, combines semantic and motion backbones, which pass their outputs to a final region segmentation network. The semantic backbone is trained in a class-agnostic manner in order to generalise to object classes beyond the training data. The core of the motion branch is a geometrical cost volume computed from optical flow, optical expansion, mono-depth and the estimated camera motion. Evaluation of the proposed architecture on the instance motion segmentation and binary moving-static segmentation problems on KITTI, DAVIS-Moving and YTVOSMoving datasets shows that the proposed method achieves state-of-the-art results on all the datasets and is able to generalise well to various environments. For the KITTI dataset, we provide an upgraded instance motion segmentation annotation which covers all moving objects. Dataset, code and models are available on the github project page github.com/michalneoral/Raptor.

Two optical flow estimation problems are addressed: (i) occlusion estimation and handling, and (ii) estimation from image sequences longer than two frames. The proposed ContinualFlow method estimates occlusions before flow, avoiding the use of flow corrupted by occlusions for their estimation. We show that providing occlusion masks as an additional input to flow estimation improves the standard performance metric by more than 25% on both KITTI and Sintel. As a second contribution, a novel method for incorporating information from past frames into flow estimation is introduced. The previous frame flow serves as an input to occlusion estimation and as a prior in occluded regions, i.e. those without visual correspondences. By continually using the previous frame flow, ContinualFlow performance improves further by 18% on KITTI and 7% on Sintel, achieving top performance on KITTI and Sintel. © 2019, Springer Nature Switzerland AG.

In this paper, we propose several improvements of the Object Scene Flow (OSF) algorithm [14]. The OSF does not use the scene flow estimated in previous frame nor the object labels and their corresponding object motion information. The goal of this paper is to use this information in order to produce temporarily consistent output throughout the whole video sequence. We evaluate the progress on the KITTI’15 multiframe dataset. We show that propagating the labels and the corresponding motion information using the estimated flow reduces the false negative rate (missed cars). Together with two further proposed improvements the overall reduction of false negative is 42%. The proposed improvements also reduce EPE on the KITTI’15 scene flow from 10.63% to 9.65%.