In this paper, we propose MoDGS, a new pipeline to render novel-view images in dynamic scenes using only casually captured monocular videos. Previous monocular dynamic NeRF or Gaussian Splatting methods strongly rely on the rapid movement of input cameras to construct multiview consistency but fail to reconstruct dynamic scenes on casually captured input videos whose cameras are static or move slowly. To address this challenging task, MoDGS adopts recent single-view depth estimation methods to guide the learning of the dynamic scene. Then, a novel 3D-aware initialization method is proposed to learn a reasonable deformation field and a new robust depth loss is proposed to guide the learning of dynamic scene geometry. Comprehensive experiments demonstrate that MoDGS is able to render high-quality novel view images of dynamic scenes from just a casually captured monocular video, which outperforms baseline methods by a significant margin.
在本文中，我们提出了 MoDGS，这是一种新的管道，仅使用随意捕获的单目视频来渲染动态场景中的新视角图像。之前的单目动态 NeRF 或高斯喷溅方法严重依赖输入相机的快速移动来构建多视图一致性，但无法在相机静止或缓慢移动的随意捕获视频输入上重建动态场景。为了解决这一挑战性任务，MoDGS 采用了最近的单视图深度估计方法来指导动态场景的学习。然后，提出了一种新颖的 3D 感知初始化方法，以学习合理的变形场，以及一个新的鲁棒深度损失，用于指导动态场景几何的学习。全面的实验表明，MoDGS 能够仅从随意捕获的单目视频中渲染出动态场景的高质量新视角图像，其性能显著优于基线方法。