Global Occlusion-Aware Transformer for Robust Stereo Matching

What is it about?

Despite the remarkable progress facilitated by learning-based stereo-matching algorithms, the performance in the ill-conditioned regions, such as the occluded regions, remains a bottleneck. Due to the limited receptive field, existing CNN-based methods struggle to handle these ill-conditioned regions effectively. To address this issue, this paper introduces a novel attention-based stereo-matching network called Global Occlusion-Aware Transformer (GOAT) to exploit long-range dependency and occlusion-awareness global context for disparity estimation. In the GOAT architecture, a parallel disparity and occlusion estimation module PDO is proposed to estimate the initial disparity map and the occlusion mask using a parallel attention mechanism. To further enhance the disparity estimates in the occluded regions, an occlusion-aware global aggregation module (OGA) is proposed. This module aims to refine the disparity in the occluded regions by leveraging restricted global correlation within the focus scope of the occluded areas. Extensive experiments were conducted on several public benchmark datasets including SceneFlow, KITTI 2015, and Middlebury. The results show that the proposed GOAT demonstrates outstanding performance among all benchmarks, particularly in the occluded regions.

Featured Image

Photo by Sam Albury on Unsplash

Photo by Sam Albury on Unsplash

Why is it important?

We explore employing restricted global spatial correlation information for stereo-matching and propose a the novel stereo-matching network named GOAT, which enables robust disparity estimation, particularly in ill-conditioned regions. • We propose a parallel disparity and occlusion estimation module (PDO) that utilizes a parallel attention mechanism to generate both disparity and occlusion masks robustly, without mutual interference. • We also propose an occlusion-aware global aggregation module (OGA) that aggregates features with a focus scope in occluded regions using self-attention, boosting disparity estimation in occluded areas.