What is it about?
The Landscape is a viewport prediction network for point cloud video streaming. Combining novel video content saliency detection technology and head trajectory prediction technology for users, Landscape performs powerful viewport prediction. In particular, to better capture the salient regions of point cloud video, Landscape leverages both static and dynamic video information to perform spatial-temporal saliency detection. With continuous video content and feedback from the user's historical head trajectory, it can accurately predict the user's viewport at the next moment. Landscape also smartly samples the point cloud video to reduce the computation burden while maintaining the video features. This will make a significant contribution to point cloud video delivery, especially by reducing bandwidth pressure by separating the quality of video content delivered inside and outside the viewport.
Featured Image
Photo by XR Expo on Unsplash
Why is it important?
Viewport prediction can greatly facilitate the point cloud video streaming, and at the same time, viewport prediction in point cloud video is difficult due to its high degree of freedom provided. As a pioneering work in this area, we use new sampling method, spatial and temporal saliency detection methods and efficiently fuse the information we can use to obtain more accurate prediction results over comparision schemes.
Perspectives
6 degrees of freedom (6DoF) video is a crucial technology for various applications such as metaverse and entertainment, and viewpoint prediction is an exceptionally unique and important research direction within it. This preliminary work on viewpoint prediction for 6DoF point cloud video effectively utilizes video and user information, with the hope of attracting further future research in this field. We hope that future research can build on our point cloud video viewport prediction work and can further improve the point cloud video delivery mechanism, allowing for a higher quality and higher speed video viewing experience on lighter quality VR glasses.
Zhi Liu
Read the Original
This page is a summary of: Demo: Landscape: Saliency and Trajectory based Viewport Prediction in Point Cloud Video Streaming, June 2023, ACM (Association for Computing Machinery),
DOI: 10.1145/3581791.3597294.
You can read the full text:
Resources
Contributors
The following have contributed to this page
