Skip to content

Latest commit

 

History

History
87 lines (66 loc) · 3.58 KB

README.md

File metadata and controls

87 lines (66 loc) · 3.58 KB
Raw

3D Face Reconstruction with StyleGAN3-based Multi-View Images and 3DDFA based Mesh Generation

3D model generation from a single image is a challenging task due to the lack of texture information and limited training data. This model proposes a novel approach for texture estimation from a single image using a generative adversarial network (StyleGAN3) and 3D Dense Face Alignment (3DDFA). The method begins by generating multi-view faces using the latent space of StyleGAN3 using Restyle encoder. Then 3DDFA generate a high-resolution texture map and map it to 3D model that is consistent with the estimated face shape.

GitHub license Maintenance

workflow

Paper Released here

Update v1.0

  1. Cleaned up some dependency issues.
  2. Made a standalone argument based code main.py. Just install the requirements push the arguments and go!

Argument example:

python main.py --input "input_data/00012.png" --pose-min -3 --pose-max 5 --center-pose 1 --output-dir "output" --show-3d

Requirements

  1. Ubuntu 22.04
  2. Python 3.8
  3. PyTorch(2.0 works great)
  4. OpenCV
  5. imageio
  6. Dlib
  7. Cython
  8. Cmake
  9. pyrallis
  10. onnx
  11. onnxruntime
  12. Open3D (for 3D visualization)

StyleGAN3 Encoder

Download the pretrained encoder from the following links and keep it on pretrained_model folder

Encoder Description
ReStyle-pSp Human Faces ReStyle-pSp trained on the FFHQ dataset over the StyleGAN3 generator.
ReStyle-e4e Human Faces ReStyle-e4e trained on the FFHQ dataset over the StyleGAN3 generator.

3DDFA Weights

Please go to the following directory:

3D-GANTex/ThreeDDFA_weights/weights

and follow the instructions in readme.md

Usage

  1. Clone the Repo:
git clone https://github.com/rohit7044/3DGANTex
  1. Download both the pretrained models mentioned above
  2. Build the cython version of NMS, Sim3DR, and the faster mesh render on the main directory
sh ./TDDFA_build.sh
  1. Open 3D-GANTex.py and make the changes mentioned on the code inside
  2. Finally after making the changes, upon running the code you will get multi_view,texture map and 3d model and it will be saved in output_data

Special Thanks

Important Note

  1. The 3D face model has uv texture embedded but it only shows the texture on Meshlab and Open3D
  2. Weak results on images with glasses.
  3. Better to take portrait image that has only the face like the example mentioned in input_data directory.