What is it about?
This research introduces the first comprehensive image quality assessment database specifically designed for 3D Gaussian Splatting (3DGS), a cutting-edge technology for creating photorealistic 3D scenes. While 3DGS produces stunning visual results, it requires massive storage space, making compression essential for practical use. However, there hasn't been a systematic way to evaluate how compression affects visual quality. Our database, 3DGS-IEval-15K, contains 15,200 images generated from 10 diverse real-world scenes using six mainstream compression methods. We carefully selected 20 viewpoints for each scene, including both typical training perspectives and challenging novel views. Through controlled experiments with 60 human viewers, we collected over 228,000 quality ratings to understand how people perceive compressed 3DGS images. Additionally, we tested 30 different quality assessment algorithms, including traditional metrics, deep learning models, and large language models, to establish comprehensive performance benchmarks. This work provides essential resources for developing better compression techniques and quality assessment tools for 3DGS technology.
Featured Image
Why is it important?
This work addresses three critical gaps in 3DGS research that have hindered practical deployment: Unprecedented Scale: Existing datasets contain fewer than 500 samples, insufficient for training robust quality assessment models. Our dataset provides 15,200 samples—over 30 times larger—enabling comprehensive model development for the first time. Systematic Compression Design: While compression is essential for 3DGS applications, previous datasets often lack systematic distortion design. We uniquely incorporate six mainstream compression algorithms with carefully designed compression levels, covering both geometric and color distortions that reflect real-world deployment scenarios. View-Dependent Quality Analysis: Unlike existing video-focused evaluations, we pioneer image-based assessment with strategic viewpoint selection. Our novel approach identifies both representative training views and challenging test views, revealing significant quality variations between perspectives—a unique 3DGS characteristic never systematically studied before. Timely Impact: As 3DGS rapidly becomes the preferred technology over NeRF for real-time rendering, this database arrives at a crucial moment when the field urgently needs specialized evaluation tools to optimize compression strategies and guide practical implementation.
Perspectives
As researchers deeply involved in 3DGS technology development, we've observed a significant disconnect between the rapid advancement of compression algorithms and the lack of proper evaluation frameworks. Through building this database, we gained several valuable insights: The most striking finding was the quality gap between training and novel viewpoints—test views consistently scored 1-2 points lower on our 10-point scale. This reveals that current 3DGS methods may be "overfitting" to training perspectives, suggesting future optimization should explicitly consider view-dependent quality distribution rather than treating all viewpoints equally. We were also surprised by the performance variability of large language models for quality assessment. While some achieved relatively competitive results (Q-Align: 0.77 correlation), others failed completely (Llama3.2-Vision: 0.07 correlation). This inconsistency highlights both the promise and current limitations of applying general-purpose vision models to specialized quality assessment tasks. Looking forward, we believe this database will catalyze development of 3DGS-specific quality metrics that better align with human perception. The systematic distortion design and large scale make it particularly valuable for training next-generation assessment models. We hope this work not only serves as an evaluation benchmark but also provides insights for optimizing 3DGS compression algorithms toward more efficient, perceptually-aware approaches that make this technology truly practical for real-world applications.
Yuke Xing
Read the Original
This page is a summary of: 3DGS-IEval-15K: A Large-scale Image Quality Evaluation Database for 3D Gaussian-Splatting, October 2025, ACM (Association for Computing Machinery),
DOI: 10.1145/3746027.3758206.
You can read the full text:
Contributors
The following have contributed to this page
