Implicit neural representation has paved the way for new approaches to dynamic scene reconstruction and rendering. Nonetheless, cutting-edge dynamic neural rendering methods rely heavily on these implicit representations, which frequently struggle to capture the intricate details of objects in the scene. Furthermore, implicit methods have difficulty achieving real-time rendering in general dynamic scenes, limiting their use in a variety of tasks. To address the issues, we propose a deformable 3D Gaussians Splatting method that reconstructs scenes using 3D Gaussians and learns them in canonical space with a deformation field to model monocular dynamic scenes. We also introduce an annealing smoothing training mechanism with no extra overhead, which can mitigate the impact of inaccurate poses on the smoothness of time interpolation tasks in real-world datasets. Through a differential Gaussian rasterizer, the deformable 3D Gaussians not only achieve higher rendering quality but also real-time rendering speed. Experiments show that our method outperforms existing methods significantly in terms of both rendering quality and speed, making it well-suited for tasks such as novel-view synthesis, time interpolation, and real-time rendering.
隐式神经表示为动态场景重建和渲染开辟了新的方法。然而，尖端的动态神经渲染方法严重依赖这些隐式表示，这些表示通常难以捕捉场景中物体的复杂细节。此外，隐式方法通常难以实现一般动态场景的实时渲染，限制了它们在各种任务中的使用。为了解决这些问题，我们提出了一种可变形3D高斯溅射方法，该方法使用3D高斯分布重建场景，并在规范空间中学习它们，配合变形场来模拟单目动态场景。我们还引入了一个无额外开销的退火平滑训练机制，可以缓解不准确姿态对实际数据集中时间插值任务平滑性的影响。通过差分高斯光栅化器，可变形3D高斯不仅实现了更高的渲染质量，还实现了实时渲染速度。实验表明，我们的方法在渲染质量和速度方面都显著优于现有方法，非常适合新视角合成、时间插值和实时渲染等任务。