What is it about?
We apply reinforcement learning to video compressive sensing to adapt the compression ratio. Specifically, video snapshot compressive imaging (SCI), which captures high-speed video using a low-speed camera, is considered in this work, in which multiple (B) video frames can be reconstructed from a snapshot measurement. One research gap in previous studies is how to adapt B in the video SCI system for different scenes. In this paper, we fill this gap by utilizing reinforcement learning (RL). An RL model, as well as various convolutional neural networks for reconstruction, are learned to achieve adaptive sensing of video SCI systems. Furthermore, the performance of an object detection network using directly the video SCI measurements without reconstruction is also used to perform RL-based adaptive video compressive sensing. Our proposed adaptive SCI method can thus be implemented at low cost and in real-time. Our work takes the technology one step further towards real applications of video SCI.
Photo by Remy Gieling on Unsplash
Why is it important?
As to the connected and autonomous vehicles, since the driving scenes being captured are dynamic and vehicle speed varies over time, it is imperative to realize adaptive video SCI for real-world applications, i.e., automatically determining the optimal B (B refers to the compression ratio) under different application environments. Take measurement-based object detection as an example, when the vehicle is driving under a slow-motion scenario, increasing B can further accelerate inference speed while still guaranteeing a high measurement-based detection accuracy. In contrast, if the driving environment changes rapidly, e.g., a red traffic light or an accident suddenly halts all vehicles, B should be decreased to avoid missing high-speed information. Therefore, it is the right time to take the developments further and make SCI systems suitable for real applications.
Read the Original
This page is a summary of: Reinforcement Learning for Adaptive Video Compressive Sensing, ACM Transactions on Intelligent Systems and Technology, July 2023, ACM (Association for Computing Machinery),
You can read the full text:
The following have contributed to this page