Novel view synthesis from unconstrained in-the-wild image collections remains a significant yet challenging task due to photometric variations and transient occluders that complicate accurate scene reconstruction. Previous methods have approached these issues by integrating per-image appearance features embeddings in Neural Radiance Fields (NeRFs). Although 3D Gaussian Splatting (3DGS) offers faster training and real-time rendering, adapting it for unconstrained image collections is non-trivial due to the substantially different architecture. In this paper, we introduce Splatfacto-W, an approach that integrates per-Gaussian neural color features and per-image appearance embeddings into the rasterization process, along with a spherical harmonics-based background model to represent varying photometric appearances and better depict backgrounds. Our key contributions include latent appearance modeling, efficient transient object handling, and precise background modeling. Splatfacto-W delivers high-quality, real-time novel view synthesis with improved scene consistency in in-the-wild scenarios. Our method improves the Peak Signal-to-Noise Ratio (PSNR) by an average of 5.3 dB compared to 3DGS, enhances training speed by 150 times compared to NeRF-based methods, and achieves a similar rendering speed to 3DGS.
从无约束的自然图像集合中进行新视角合成一直是一个重大且具有挑战性的任务，因为光度变化和瞬时遮挡物使得准确的场景重建变得复杂。先前的方法通过在神经辐射场（NeRFs）中集成每张图片的外观特征嵌入来处理这些问题。尽管三维高斯投影（3DGS）提供了更快的训练速度和实时渲染，但由于架构差异显著，将其适用于无约束的图像集合并非易事。在本文中，我们介绍了一种名为Splatfacto-W的方法，该方法将每个高斯的神经颜色特征和每张图像的外观嵌入整合到光栅化过程中，并结合球谐函数背景模型来表示变化的光度外观和更好地描述背景。我们的主要贡献包括潜在外观建模、高效的瞬时物体处理和精确的背景建模。Splatfacto-W提供了高质量的实时新视角合成，改善了在自然场景中的场景一致性。我们的方法与3DGS相比，平均提高了5.3 dB的峰值信噪比（PSNR），训练速度比基于NeRF的方法快150倍，并且达到了与3DGS类似的渲染速度。