Skip to content

Latest commit

 

History

History
81 lines (64 loc) · 2.61 KB

BENCHMARK_iPhone.md

File metadata and controls

81 lines (64 loc) · 2.61 KB
Raw

Benchmark: DyCheck iPhone Dataset

Table of Contents

1 Download Data

# this environment variable is used for demonstration
cd /path/to/this/repo
export PGDVS_ROOT=$PWD

Please follow DyCheck's official tutorial to download the iPhone dataset to ${PGDVS_ROOT}/data. After this, you should have a structure as

.
+-- data
|  +-- iphone
|  |  +-- apple
|  |  +-- block
|  |  ...

2 Preprocess

2.1 Use Precomputed Flow and Mask

gdown 1SgvqDJcuFaGJr6Lr3bE9B-knjbOnADQs -O ${PGDVS_ROOT}/data/
unzip ${PGDVS_ROOT}/data/dycheck_iphone_flow_mask.zip -d ${PGDVS_ROOT}/data

2.2 Reproduce Preprocessed Results

Our pseudo-generalized approach requires optical flow and mask for potentially dynamic content. For this, we need several third parties's repositories and checkpoints. NOTE: the CUDA_HOME must be set correctly for detectron2's installation and consequentially OneFormer's usage.

CUDA_HOME=/usr/local/cuda  # set to your own CUDA_HOME, where nvcc is installed
bash ${PGDVS_ROOT}/scripts/preprocess/preprocess.sh \
  ${CUDA_HOME} \
  ${PGDVS_ROOT} \
  ${PGDVS_ROOT}/data \
  prepare

After running the command, repositories and pretrained checkpoints will be saved to ${PGDVS_ROOT}/third_parties and ${PGDVS_ROOT}/ckpts respectively.

We then compute optical flow and mask with

cd ${PGDVS_ROOT}

bash ${PGDVS_ROOT}/scripts/preprocess/preprocess.sh \
  /usr/local/cuda/ \
  ${PGDVS_ROOT} \
  ${PGDVS_ROOT}/data/iphone \
  execute_on_dycheck \
  ${PGDVS_ROOT}/data/dycheck_iphone_flow_mask \
  apple  # can be one of [apple block paper-windmill space-out spin teddy wheel]

The computed optical flows and masks will be saved to ${PGDVS_ROOT}/data/dycheck_iphone_flow_mask

3 Run Benchmark

To obtain quantitative results, run the following command. All results will be saved to ${PGDVS_ROOT}/experiments.

benchmark_type=st_gnt_masked_attn_dy_cvd_pcl_clean_render_point
scene_id='[apple]'  # or 'null' to evaluate on all scenes

bash ${PGDVS_ROOT}/scripts/benchmark.sh \
  ${PGDVS_ROOT} \
  ${PGDVS_ROOT}/ckpts \
  ${PGDVS_ROOT}/data \
  dycheck_iphone \
  ${scene_id} \
  ${benchmark_type}