Skip to content

YueWuHKUST/ICCV2021-Embedding-Novel-Views-in-a-Single-JPEG-Image

Repository files navigation

ICCV2021-Embedding-Novel-Views-in-a-Single-JPEG-Image

This is the official implmentation of ICCV 2021 paper: Embedding-Novel-Views-in-a-Single-JPEG-Image

Environment

please refer to the enviroment.yml

Dataset

We provide the test set we use in the Google drive. We randomly select 1500 sequences from RealEstate10K to generate MPIs for testing. The IDs of the sequences for training and testing are in train_data_ids.txt and test_data_ids.txt respectively.

And the stereo_mpi represents the MPIs generated using Stereo Magnification. And the pb_mpi_32 represents the MPIs generated using the Pushing the Boundaries of View Extrapolation with Multiplane Images. The original output of this method generate 128-layer MPIs, we merge the 128-layer MPIs to 32-layer due to network capacity limitation.

We use their public code for the MPIs generations. For the details of how the MPIs are generated, please refer to their public repo.

The Folder structure

ICCV2021-Embedding-Novel-Views-in-a-Single-JPEG-Image/
│
└─── datasets/
    |
    └─── stereo_mpi/
    │
        └─── test_final/   # the test dataset of stereo-magnification
	|
	└─── pb_mpi_32/  # the test dataset of pb_mpi, we merge the original 128 mpi to 32 layers
	    |
	    └─── test_final /
└─── checkpoints/
    |
    └─── stereo_final
│
        └─── models
│
            └─── decoder_003_00015000.pth
│
            └─── discriminator_003_00015000.pth
│
            └─── encoder_003_00015000.pth
    ....
    test.py
    test.sh
    ....

The pretrained models

The pretrained checkpoints are shared in the Google drive link as follows:

Methods Links
Stereo magnification Google drive
PB-MPI Google drive
LLFF Google drive

How to test

After put the test data and pretrained checkpoints in the corresponding folder,

Step1:

./test.sh

Run the test.sh to generate the embedding image and predicted MPIs.

The results will be stored in './checkpoints/[checkpoint_name]/test_result/

Step2:

./render.sh

Run the render.sh to render novel views using predicted MPIs. The results will be stored in './checkpoints/[checkpoint_name]/render/

To do

  • release training
  • Write document and instructions

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published