Skip to content

Latest commit

 

History

History
5 lines (3 loc) · 3.48 KB

2406.15643.md

File metadata and controls

5 lines (3 loc) · 3.48 KB

Taming 3DGS: High-Quality Radiance Fields with Limited Resources

3D Gaussian Splatting (3DGS) has transformed novel-view synthesis with its fast, interpretable, and high-fidelity rendering. However, its resource requirements limit its usability. Especially on constrained devices, training performance degrades quickly and often cannot complete due to excessive memory consumption of the model. The method converges with an indefinite number of Gaussians -- many of them redundant -- making rendering unnecessarily slow and preventing its usage in downstream tasks that expect fixed-size inputs. To address these issues, we tackle the challenges of training and rendering 3DGS models on a budget. We use a guided, purely constructive densification process that steers densification toward Gaussians that raise the reconstruction quality. Model size continuously increases in a controlled manner towards an exact budget, using score-based densification of Gaussians with training-time priors that measure their contribution. We further address training speed obstacles: following a careful analysis of 3DGS' original pipeline, we derive faster, numerically equivalent solutions for gradient computation and attribute updates, including an alternative parallelization for efficient backpropagation. We also propose quality-preserving approximations where suitable to reduce training time even further. Taken together, these enhancements yield a robust, scalable solution with reduced training times, lower compute and memory requirements, and high quality. Our evaluation shows that in a budgeted setting, we obtain competitive quality metrics with 3DGS while achieving a 4--5x reduction in both model size and training time. With more generous budgets, our measured quality surpasses theirs. These advances open the door for novel-view synthesis in constrained environments, e.g., mobile devices.

3D 高斯散射(3DGS)通过其快速、可解释和高保真渲染,已转变了新视角合成的方式。然而,其资源需求限制了其可用性。特别是在资源受限的设备上,训练性能迅速下降,常常因模型的过度内存消耗而无法完成训练。该方法使用不定数量的高斯核,许多高斯核是多余的,这使得渲染过程不必要地缓慢,并阻碍了其在需要固定大小输入的下游任务中的使用。为了解决这些问题,我们应对了在预算内训练和渲染 3DGS 模型的挑战。我们使用一个引导性的、纯粹建设性的密集化过程,引导密集化过程向提高重建质量的高斯核倾斜。模型大小在控制中不断增加,向精确预算逼近,使用基于分数的高斯核密集化以及训练时先验来衡量它们的贡献。我们还解决了训练速度的障碍:在对 3DGS 原始流程进行仔细分析后,我们得出了用于梯度计算和属性更新的更快、数值等效的解决方案,包括一个用于高效反向传播的替代并行化方案。我们还提出了适当的保质近似方法,以进一步减少训练时间。综合这些增强功能,我们提供了一个健壮、可扩展的解决方案,具有更短的训练时间、更低的计算和内存需求,以及高质量。我们的评估显示,在有预算的情况下,我们与 3DGS 的质量指标具有竞争力,同时实现了模型大小和训练时间的 4-5 倍减少。在更宽松的预算下,我们的测量质量超过了他们。这些进步为在受限环境中(例如移动设备)的新视角合成打开了大门。