CHASE: 3D-Consistent Human Avatars with Sparse Inputs via Gaussian Splatting and Contrastive Learning
Recent advancements in human avatar synthesis have utilized radiance fields to reconstruct photo-realistic animatable human avatars. However, both NeRFs-based and 3DGS-based methods struggle with maintaining 3D consistency and exhibit suboptimal detail reconstruction, especially with sparse inputs. To address this challenge, we propose CHASE, which introduces supervision from intrinsic 3D consistency across poses and 3D geometry contrastive learning, achieving performance comparable with sparse inputs to that with full inputs. Following previous work, we first integrate a skeleton-driven rigid deformation and a non-rigid cloth dynamics deformation to coordinate the movements of individual Gaussians during animation, reconstructing basic avatar with coarse 3D consistency. To improve 3D consistency under sparse inputs, we design Dynamic Avatar Adjustment(DAA) to adjust deformed Gaussians based on a selected similar pose/image from the dataset. Minimizing the difference between the image rendered by adjusted Gaussians and the image with the similar pose serves as an additional form of supervision for avatar. Furthermore, we propose a 3D geometry contrastive learning strategy to maintain the 3D global consistency of generated avatars. Though CHASE is designed for sparse inputs, it surprisingly outperforms current SOTA methods in both full and sparse settings on the ZJU-MoCap and H36M datasets, demonstrating that our CHASE successfully maintains avatar's 3D consistency, hence improving rendering quality.
最近在人类化身合成方面的进展利用辐射场来重建可动画的逼真化身。然而，基于NeRFs和3DGS的方法在保持3D一致性方面存在困难，并且在稀疏输入的情况下，细节重建效果欠佳。为了解决这一挑战，我们提出了CHASE，它引入了来自跨姿态的内在3D一致性监督和3D几何对比学习，从而在稀疏输入情况下实现了与完整输入相媲美的性能。借鉴以往的研究，我们首先结合了骨架驱动的刚性变形和非刚性布料动态变形，以协调动画中各个高斯的运动，从而重建具有粗略3D一致性的基础化身。为了在稀疏输入下提高3D一致性，我们设计了动态化身调整（DAA），根据数据集中选择的相似姿态/图像来调整变形后的高斯。通过最小化由调整后的高斯渲染的图像与相似姿态的图像之间的差异，作为对化身的额外监督。此外，我们提出了一种3D几何对比学习策略，以保持生成化身的3D全局一致性。尽管CHASE是为稀疏输入设计的，但它在ZJU-MoCap和H36M数据集的完整和稀疏设置中均优于当前的最先进方法，这表明我们的CHASE成功保持了化身的3D一致性，从而提高了渲染质量。