3D Gaussian Splatting (3DGS) has become an emerging technique with remarkable potential in 3D representation and image rendering. However, the substantial storage overhead of 3DGS significantly impedes its practical applications. In this work, we formulate the compact 3D Gaussian learning as an end-to-end Rate-Distortion Optimization (RDO) problem and propose RDO-Gaussian that can achieve flexible and continuous rate control. RDO-Gaussian addresses two main issues that exist in current schemes: 1) Different from prior endeavors that minimize the rate under the fixed distortion, we introduce dynamic pruning and entropy-constrained vector quantization (ECVQ) that optimize the rate and distortion at the same time. 2) Previous works treat the colors of each Gaussian equally, while we model the colors of different regions and materials with learnable numbers of parameters. We verify our method on both real and synthetic scenes, showcasing that RDO-Gaussian greatly reduces the size of 3D Gaussian over 40x, and surpasses existing methods in rate-distortion performance.
3D 高斯涂抹（3DGS）已成为3D表征和图像渲染中一种具有显著潜力的新兴技术。然而，3DGS的巨大存储开销显著阻碍了其实际应用。在这项工作中，我们将紧凑的3D高斯学习表述为端到端的速率失真优化（RDO）问题，并提出了RDO-Gaussian，能够实现灵活和连续的速率控制。RDO-Gaussian解决了当前方案中存在的两个主要问题：1）与之前只是在固定失真下最小化速率的尝试不同，我们引入了动态剪枝和熵约束向量量化（ECVQ），同时优化速率和失真。2）以往的工作对每个高斯的颜色处理相同，而我们对不同区域和材料的颜色采用可学习的参数数量进行建模。我们在真实和合成场景中验证了我们的方法，显示出RDO-Gaussian在减少3D高斯大小方面超过40倍，并且在速率失真表现上超越了现有方法。