Representing and rendering dynamic scenes has been an important but challenging task. Especially, to accurately model complex motions, high efficiency is usually hard to guarantee. To achieve real-time dynamic scene rendering while also enjoying high training and storage efficiency, we propose 4D Gaussian Splatting (4D-GS) as a holistic representation for dynamic scenes rather than applying 3D-GS for each individual frame. In 4D-GS, a novel explicit representation containing both 3D Gaussians and 4D neural voxels is proposed. A decomposed neural voxel encoding algorithm inspired by HexPlane is proposed to efficiently build Gaussian features from 4D neural voxels and then a lightweight MLP is applied to predict Gaussian deformations at novel timestamps. Our 4D-GS method achieves real-time rendering under high resolutions, 82 FPS at an 800×800 resolution on an RTX 3090 GPU while maintaining comparable or better quality than previous state-of-the-art methods.
表示和渲染动态场景一直是一个重要但具有挑战性的任务。特别是，要准确地模拟复杂的运动，通常很难保证高效率。为了实现实时动态场景渲染，同时享有高训练和存储效率，我们提出了4D高斯溅射（4D-GS）作为动态场景的整体表示，而不是为每个单独的帧应用3D-GS。在4D-GS中，提出了一个包含3D高斯和4D神经体素的新型显式表示。我们受到HexPlane启发，提出了一个分解的神经体素编码算法，以有效地从4D神经体素构建高斯特征，然后应用轻量级的多层感知器（MLP）来预测新时间戳的高斯变形。我们的4D-GS方法在高分辨率下实现了实时渲染，在RTX 3090 GPU上以800×800分辨率达到82 FPS，同时保持与之前最先进方法相当或更好的质量。