Publication

Abstract

In this work we exploit segmentation to construct appearance descriptors that can robustly deal with occlusion and background changes. For this, we downplay measurements coming from areas that are unlikely to belong to the same region as the descriptor’s center, as suggested by soft segmentation masks. Our treatment is applicable to any image point, i.e. dense, and its computational overhead is in the order of a few seconds. We integrate this idea with Dense SIFT, and also with Dense Scale and Rotation Invariant Descriptors (SID), delivering descriptors that are densely computable, invariant to scaling and rotation, and robust to background changes. We apply our approach to standard benchmarks on large displacement motion estimation using SIFT-flow and widebaseline stereo, systematically demonstrating that the introduction of segmentation yields clear improvements.

Categories

pattern recognition.

Author keywords

computer vision, appearance descriptors, image segmentation, optical flow, stereo

Scientific reference

E. Trulls Fortuny, I. Kokkinos, A. Sanfeliu and F. Moreno-Noguer. Dense segmentation-aware descriptors, 2013 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2013, Portland, OR, USA, pp. 2890-2897.