RTGS: Enabling Real-Time Gaussian Splatting on Mobile Devices Using Efficiency-Guided Pruning and Foveated Rendering
Point-Based Neural Rendering (PBNR), i.e., the 3D Gaussian Splatting-family algorithms, emerges as a promising class of rendering techniques, which are permeating all aspects of society, driven by a growing demand for real-time, photorealistic rendering in AR/VR and digital twins. Achieving real-time PBNR on mobile devices is challenging. This paper proposes RTGS, a PBNR system that for the first time delivers real-time neural rendering on mobile devices while maintaining human visual quality. RTGS combines two techniques. First, we present an efficiency-aware pruning technique to optimize rendering speed. Second, we introduce a Foveated Rendering (FR) method for PBNR, leveraging humans' low visual acuity in peripheral regions to relax rendering quality and improve rendering speed. Our system executes in real-time (above 100 FPS) on Nvidia Jetson Xavier board without sacrificing subjective visual quality, as confirmed by a user study.
本文介绍了一种名为RTGS的点云神经渲染（PBNR）系统，这是首个在移动设备上实现实时神经渲染并保持人类视觉质量的系统。PBNR是一类新兴的渲染技术，特别适用于增强现实/虚拟现实和数字孪生等领域，因其实时性和逼真度需求日益增长而备受关注。在移动设备上实现实时PBNR具有挑战性。 RTGS系统结合了两种技术。首先，我们提出了一种效率感知的修剪技术，以优化渲染速度。其次，我们引入了一种称为焦点渲染（FR）的方法，用于PBNR，利用人眼在周边区域的低视觉敏感度，以降低渲染质量要求并提高渲染速度。我们的系统在Nvidia Jetson Xavier板上能够实现实时运行（超过100帧每秒），同时未牺牲主观视觉质量，这一点由用户研究得到了确认。