3D reconstruction and simulation, while interrelated, have distinct objectives: reconstruction demands a flexible 3D representation adaptable to diverse scenes, whereas simulation requires a structured representation to model motion principles effectively. This paper introduces the Mesh-adsorbed Gaussian Splatting (MaGS) method to resolve such a dilemma. MaGS constrains 3D Gaussians to hover on the mesh surface, creating a mutual-adsorbed mesh-Gaussian 3D representation that combines the rendering flexibility of 3D Gaussians with the spatial coherence of meshes. Leveraging this representation, we introduce a learnable Relative Deformation Field (RDF) to model the relative displacement between the mesh and 3D Gaussians, extending traditional mesh-driven deformation paradigms that only rely on ARAP prior, thus capturing the motion of each 3D Gaussian more precisely. By joint optimizing meshes, 3D Gaussians, and RDF, MaGS achieves both high rendering accuracy and realistic deformation. Extensive experiments on the D-NeRF and NeRF-DS datasets demonstrate that MaGS can generate competitive results in both reconstruction and simulation.
3D 重建和模拟虽然相互关联，但具有不同的目标：重建要求一种灵活的 3D 表示，适应多样化的场景，而模拟则需要一种结构化的表示以有效地模拟运动原理。本文介绍了网格吸附高斯喷溅（MaGS）方法来解决这种困境。MaGS 将 3D 高斯限制在网格表面上漂浮，创建了一个相互吸附的网格-高斯 3D 表示，结合了 3D 高斯的渲染灵活性与网格的空间连贯性。利用这种表示，我们引入了一个可学习的相对形变场（RDF），用于模拟网格和 3D 高斯之间的相对位移，扩展了传统的仅依赖 ARAP 先验的网格驱动形变范式，从而更精确地捕捉每个 3D 高斯的运动。通过联合优化网格、3D 高斯和 RDF，MaGS 实现了高渲染精度和真实的形变。在 D-NeRF 和 NeRF-DS 数据集上的广泛实验表明，MaGS 能在重建和模拟方面生成具有竞争力的结果。