The creation of high-quality 3D assets is paramount for applications in digital heritage preservation, entertainment, and robotics. Traditionally, this process necessitates skilled professionals and specialized software for the modeling, texturing, and rendering of 3D objects. However, the rising demand for 3D assets in gaming and virtual reality (VR) has led to the creation of accessible image-to-3D technologies, allowing non-professionals to produce 3D content and decreasing dependence on expert input. Existing methods for 3D content generation struggle to simultaneously achieve detailed textures and strong geometric consistency. We introduce a novel 3D content creation framework, ScalingGaussian, which combines 3D and 2D diffusion models to achieve detailed textures and geometric consistency in generated 3D assets. Initially, a 3D diffusion model generates point clouds, which are then densified through a process of selecting local regions, introducing Gaussian noise, followed by using local density-weighted selection. To refine the 3D gaussians, we utilize a 2D diffusion model with Score Distillation Sampling (SDS) loss, guiding the 3D Gaussians to clone and split. Finally, the 3D Gaussians are converted into meshes, and the surface textures are optimized using Mean Square Error(MSE) and Gradient Profile Prior(GPP) losses. Our method addresses the common issue of sparse point clouds in 3D diffusion, resulting in improved geometric structure and detailed textures. Experiments on image-to-3D tasks demonstrate that our approach efficiently generates high-quality 3D assets.
创建高质量的3D资产在数字遗产保护、娱乐和机器人技术应用中至关重要。传统上，这一过程需要专业人士和专门的软件来进行3D对象的建模、纹理贴图和渲染。然而，随着游戏和虚拟现实（VR）对3D资产需求的上升，出现了便捷的图像转3D技术，使非专业人士能够制作3D内容，从而减少对专家的依赖。现有的3D内容生成方法在同时实现详细纹理和强几何一致性方面存在困难。我们提出了一种新型的3D内容创建框架——ScalingGaussian，该框架结合了3D和2D扩散模型，以在生成的3D资产中实现详细纹理和几何一致性。首先，3D扩散模型生成点云，然后通过选择局部区域、引入高斯噪声以及局部密度加权选择的过程来加密这些点云。为了精炼3D高斯分布，我们利用带有评分蒸馏采样（SDS）损失的2D扩散模型，指导3D高斯分布进行克隆和拆分。最后，将3D高斯分布转换为网格，并使用均方误差（MSE）和梯度轮廓先验（GPP）损失优化表面纹理。我们的方法解决了3D扩散中的点云稀疏问题，改善了几何结构和纹理细节。对图像转3D任务的实验表明，我们的方法能够高效生成高质量的3D资产。