What is it about?

Sports broadcasts are usually filmed in a wide format, while many people watch on upright phones. Simply cropping the picture can lose the action. H2V4Sports converts horizontal sports video into vertical video by following important objects, such as a basketball or a diver. It combines object detection with tracking to help locate these objects when the scene becomes difficult. The system then matches objects across frames and smooths the movement of the viewing window. A virtual director chooses how to frame one or more objects, with a central view as a fallback when none is detected. We demonstrated the approach on basketball and diving. The reported test reached 24 frames per second on an NVIDIA GeForce RTX 3090, supporting live video conversion in that setup.

Featured Image

Why is it important?

Vertical viewing changes what a sports audience can see: a narrow frame must follow the important action as it moves across a wide broadcast. This work brings object detection, tracking and automated framing together in a system designed for live sport. It offers a practical route for video platforms to adapt existing horizontal coverage for smartphones. Its basketball and diving demonstrations show the approach in two different sports, while broader performance and viewing benefits still need more extensive evaluation.

Perspectives

For me, the central goal of H2V4Sports is simple: keep the important action visible when a sports broadcast is shown vertically. Following an object is only part of that task. The viewing window also needs to move smoothly and respond sensibly when several objects appear or detection fails. I see the system as a useful connection between computer vision and the practical demands of live sports presentation.

Wei Feng
Tsinghua University

Read the Original

This page is a summary of: H2V4Sports: Real-Time Horizontal-to-Vertical Video Converter for Sports Lives via Fast Object Detection and Tracking, October 2023, ACM (Association for Computing Machinery),
DOI: 10.1145/3581783.3612669.
You can read the full text:

Read

Resources

Contributors

The following have contributed to this page