Authors: Guy Erez, Ron Shapira Weber, and Oren Freifeld.
This code repository corresponds to our ECCV '22 paper: DeepMCBM: A Deep Moving-camera Background Model. DeepMCBM is a novel 2D-based method for unsupervised learning of a moving-camera background model, which is highly scalable and allows for relatively-free camera motion.
The repository is equipped with a DeepMCBM_env.yml file.
Run conda env create -f DeepMCBM_env.yml from your terminal to set a conda environment using this file.
To ensure the environment is set properly, activate the new environment and run a "dry run" with few epochs:
conda activate DeepMCBM
python src/DeepMCBM.py --DryRun
To train, predict and evaluate a deepMCBM module on the default tennis sequence:
python src/DeepMCBM.py
The default values for the input, output, and checkpoints paths are set in src/args.py and can be changed to any path you wish. The requirement for the input directory is to have the following subdirectories: "frames" include the sequence frames, and if ground truth labels are available, a "GT" directory containing the ground truth frames. See the input/tennis for an example. The output directories are named by the sequence and the log_name argument: output/sequence_name/log_name in this directory you will find:
- background_estimation directory containing the background estimation of the model.
- MSE directory containing the Mean Square Error (MSE) computed using the ground truth labels.
- panoramic_robust_mean.png image, shows the alignment result of the STN module.
You can change the log_name simply by adding log_name "my_new_name" to your command line.
To only predict and evaluate metrics:
python src/DeepMCBM.py --no_train_BMN --no_train_STN
You can change the loaded checkpoint using a flag:
python src/DeepMCBM.py --no_train_BMN --no_train_STN --BMN_ckpt ckpt_file.ckpt
Or by editing the MCBM_CKPT argument in src/args.py
Note: when using a pretrained model, the argument --pad, describing the size of the padding, must be the same as in the training phase.