Regressão Afim Espaço-Temporal para Rastreamento de Características

Abstract

Feature association is a fundamental prerequisite for visual localization pipelines. Typically, these methods rely on feature matching to estimate relative motion based on projective geometric constraints. Despite significant advances in feature association, most existing techniques rely on pairwise matching paradigms and often neglect the rich temporal context inherent in image sequences. In this thesis, we revisit the canonical Kanade-Lucas-Tomasi (KLT) feature tracker. We reformulate this classic algorithm by integrating deep neural network mechanisms for spatiotemporal and geometric learning. The proposed methodology uses a convolutional neural network trained to regress affine transformation parameters across consecutive frame patches. This capability allows for precise tracking of interest points considering the consistency of the projective geometry. To perform the training of the proposed network, we introduce a versatile protocol to synthesize feature tracking annotations from already available datasets. This methodology leverages state-of-the-art feature extraction and matching along with a model selection criterion based on epipolar geometry. Experimental evaluations on the TUM RGB-D benchmark demonstrate the consistent superiority of the proposed method in estimating relative camera motion compared to KLT and the Pips++ method. Although our method exhibits a lower inlier ratio, the resulting correspondence subset possesses significantly higher geometric fidelity. These results establish the proposed method as a robust solution, suitable for deployment in embedded systems with limited resources.