Text-driven 3D scene generation has seen significant advancements recently. However, most existing methods generate single-view images using generative models and then stitch them together in 3D space. This independent generation for each view often results in spatial inconsistency and implausibility in the 3D scenes. To address this challenge, we proposed a novel text-driven 3D-consistent scene generation model: SceneDreamer360. Our proposed method leverages a text-driven panoramic image generation model as a prior for 3D scene generation and employs 3D Gaussian Splatting (3DGS) to ensure consistency across multi-view panoramic images. Specifically, SceneDreamer360 enhances the fine-tuned Panfusion generator with a three-stage panoramic enhancement, enabling the generation of high-resolution, detail-rich panoramic images. During the 3D scene construction, a novel point cloud fusion initialization method is used, producing higher quality and spatially consistent point clouds. Our extensive experiments demonstrate that compared to other methods, SceneDreamer360 with its panoramic image generation and 3DGS can produce higher quality, spatially consistent, and visually appealing 3D scenes from any text prompt.
基于文本的 3D 场景生成最近取得了显著进展。然而，大多数现有方法生成单视图图像并将其拼接到 3D 空间中，这种对每个视图的独立生成常常导致 3D 场景中的空间不一致和不现实。为了解决这个问题，我们提出了一种新型的基于文本的 3D 一致场景生成模型：SceneDreamer360。我们的方法利用基于文本的全景图像生成模型作为 3D 场景生成的先验，并采用 3D 高斯点云（3DGS）确保多视图全景图像的一致性。具体来说，SceneDreamer360 对细化的 Panfusion 生成器进行了三阶段的全景增强，实现了高分辨率、细节丰富的全景图像生成。在 3D 场景构建过程中，我们使用了一种新型的点云融合初始化方法，生成了更高质量和空间一致的点云。我们的大量实验表明，与其他方法相比，SceneDreamer360 通过全景图像生成和 3DGS 能够从任何文本提示生成更高质量、空间一致且视觉上更吸引人的 3D 场景。