Helping AI understand 360-degree scenes when the camera turns

What is it about?

A 360-degree camera captures an entire scene, but an AI model can give different answers when the same panorama is rotated. We introduce EquiSSL, a method that helps panoramic vision models recognise the same objects and surfaces consistently across camera orientations. It makes the position information used inside the model less dependent on local coordinate directions and aligns the learning targets of rotated views. The model can then learn from unlabelled panoramas before being trained with a small set of labelled images. On Stanford2D3D validation images, the model's segmentation score fell by an average of 0.8% relative to upright views when tested with random rotations up to 90 degrees.

Featured Image

Photo by Remy Gieling on Unsplash

Photo by Remy Gieling on Unsplash

Why is it important?

Panoramic cameras and headsets move and tilt during use, so scene understanding should not depend on keeping every image upright. EquiSSL combines rotation stability with learning from limited labelled data, without increasing the underlying vision model's parameter count. This offers a practical direction for more consistent panoramic perception in immersive video, virtual and augmented reality, and image-based lighting. The study also shows why matching the geometry of a model and its self-supervised training targets matters.

Perspectives