Skip to content

Latest commit

 

History

History
5 lines (3 loc) · 2.48 KB

2405.17835.md

File metadata and controls

5 lines (3 loc) · 2.48 KB

Deform3DGS: Flexible Deformation for Fast Surgical Scene Reconstruction with Gaussian Splatting

Tissue deformation poses a key challenge for accurate surgical scene reconstruction. Despite yielding high reconstruction quality, existing methods suffer from slow rendering speeds and long training times, limiting their intraoperative applicability. Motivated by recent progress in 3D Gaussian Splatting, an emerging technology in real-time 3D rendering, this work presents a novel fast reconstruction framework, termed Deform3DGS, for deformable tissues during endoscopic surgery. Specifically, we introduce 3D GS into surgical scenes by integrating a point cloud initialization to improve reconstruction. Furthermore, we propose a novel flexible deformation modeling scheme (FDM) to learn tissue deformation dynamics at the level of individual Gaussians. Our FDM can model the surface deformation with efficient representations, allowing for real-time rendering performance. More importantly, FDM significantly accelerates surgical scene reconstruction, demonstrating considerable clinical values, particularly in intraoperative settings where time efficiency is crucial. Experiments on DaVinci robotic surgery videos indicate the efficacy of our approach, showcasing superior reconstruction fidelity PSNR: (37.90) and rendering speed (338.8 FPS) while substantially reducing training time to only 1 minute/scene.

组织变形对于精确的手术场景重建构成了关键挑战。尽管现有方法能够实现高质量的重建,但它们的渲染速度慢,训练时间长,限制了它们在手术中的适用性。受近期在实时三维渲染技术——三维高斯喷溅(3DGS)进展的启发,本工作提出了一种名为 Deform3DGS 的新型快速重建框架,用于内窥镜手术中的可变形组织。具体来说,我们通过整合点云初始化来将3D GS 引入手术场景中,以改善重建。此外,我们提出了一种新颖的灵活变形建模方案(FDM),用于在个别高斯的水平上学习组织变形动态。我们的 FDM 能够使用高效的表示来模拟表面变形,允许实时渲染性能。更重要的是,FDM 显著加速了手术场景重建,显示出相当的临床价值,特别是在时间效率至关重要的手术中设置中。在 DaVinci 机器人手术视频上的实验表明了我们方法的有效性,展示了优越的重建保真度 PSNR:(37.90)和渲染速度(338.8 FPS),同时将训练时间大幅缩短到只有1分钟/场景。