What is it about?
In this paper, we introduce a novel approach to address the challenge of effectively utilizing both RGB and depth information for semantic segmentation. Our approach, Intra-inter Modal Attention (IMA) blocks, considers both intra-modal and inter-modal aspects of the information to produce better results than prior methods which primarily focused on inter-modal relationships.
Featured Image
Photo by Raymond Hsu on Unsplash
Why is it important?
Our proposed method, which takes into account both intra- and inter-modal characteristics, results in improved segmentation accuracy compared to previous approaches using only intra-modal or inter-modal attention. We present an in-depth analysis of intra-modal, inter-modal, and intra-inter modal attention in RGB-D segmentation, which can serve as a useful reference for future work in the field.
Perspectives
We have introduced Intra-inter Modal Attention (IMA) blocks, a novel plug-and-play module for RGB-D semantic segmentation. The IMA blocks consist of two main modules, SIM-NL and ACF. SIM-NL captures intra- and inter-modal information at the spatial level, while ACF adaptively reweights spatially-correlated features at the channel level. Our experiments on three benchmark datasets and six different baseline models consistently showed improved performance with IMA.
Sungeun Hong
Inha University
Read the Original
This page is a summary of: Intra-inter Modal Attention Blocks for RGB-D Semantic Segmentation, June 2023, ACM (Association for Computing Machinery),
DOI: 10.1145/3591106.3592235.
You can read the full text:
Resources
Contributors
The following have contributed to this page
