Skip to content

Latest commit

 

History

History
149 lines (125 loc) · 6.94 KB

INSTRUCTIONS.md

File metadata and controls

149 lines (125 loc) · 6.94 KB
Raw

Instructions

Training

For training the complete Active Neural SLAM model on the Exploration task:

python main.py

Specifying number of threads

The code runs multiple parallel threads for training. Each thread loads a scene on a GPU. The code automatically decides the total number of threads and number of threads on each GPU based on the available GPUs.

If you would like to not use the auto gpu config, you need to specify the following:

-n, --num_processes NUM_PROCESSES
--num_processes_per_gpu NUM_PROCESSES_PER_GPU

NUM_PROCESSES_PER_GPU will depend on your GPU memory, 11 works well of 16GB GPUs. NUM_PROCESSES depends on the number of GPUs used for training and NUM_PROCESSES_PER_GPU such that

NUM_PROCESSES <= min(NUM_PROCESSES_PER_GPU * number of GPUs, 72)

The Gibson training set consists of 72 scenes.

For example, on a 8 GPU system, with 16GB memory per GPU:

python main.py --auto_gpu_config 0 -n 72 --num_processes_per_gpu 11 --sim_gpu_id 1

Here, sim_gpu_id = 1 specifies simulator threads to run from GPUs 1 onwards, and using GPU 0 only for pytorch model. Each GPU from 1 to 6 will run 11 threads each, and GPU 7 will run 6 threads.

Specifying log location, periodic model dumps

python main.py -d saved/ --exp_name exp1 --save_periodic 100000

The above will save the best model files and training log at saved/models/exp1/ and save all models periodically every 100000 steps at saved/dump/exp1/. Each module will be saved in a separate file.

Specifying which modules to train and load

The Active Neural SLAM model consists of three independent modules: a Global Policy, a Local Policy and a Neural SLAM Module. The model and code is modular, which means any subset of modules can be trained. Specifying which modules need to trained using

  --train_global TRAIN_GLOBAL
        0: Do not train the Global Policy
        1: Train the Global Policy (default: 1)
  --train_local TRAIN_LOCAL
        0: Do not train the Local Policy
        1: Train the Local Policy (default: 1)
  --train_slam TRAIN_SLAM
        0: Do not train the Neural SLAM Module
        1: Train the Neural SLAM Module (default: 1)

Each module can also be loaded independently using --load_global, --load_local and --load_slam arguments.

Using deterministic local policy

Instead of training the local policy, a deterministic local policy can be used which results in much faster training and slightly lower final performance. Add --use_deterministic_local 1 argument to use the deterministic local policy.

Hyper-parameters

Most of the default hyper-parameters should work fine. Some hyperparameters are set for training with 72 threads, which might need to be tuned when using fewer threads. Fewer threads lead to smaller batch size for Local and Global Policy. Consequently, their learning rates might need to be tuned:

--local_optimizer, (default='adam,lr=0.0001')
--global_lr, (default=2.5e-5)

Specifying actuation and sensor noise

The code uses actuation and sensor noise based on models trained on real-world data. To turn off the actuation and sensor noise use --noisy_actions 0 and --noisy_odometry 0, respectively.

Downloading pre-trained models

mkdir pretrained_models
wget -O pretrained_models/model_best.global http://www.cs.cmu.edu/~dchaplot/projects/active_neural_slam/model_best.global
wget -O pretrained_models/model_best.local http://www.cs.cmu.edu/~dchaplot/projects/active_neural_slam/model_best.local
wget -O pretrained_models/model_best.slam http://www.cs.cmu.edu/~dchaplot/projects/active_neural_slam/model_best.slam

Evaluation

The following are instructions to evaluate on the Gibson val set.

Converting datasets

To parallelize evaluation for speed, we provide a script to convert the Gibson val set into multi-threading format:

python scripts/convert_datasets.py

The above will create a multi-thread version of the val set at data/datasets/pointnav/gibson/v1//val_mt/.

Specifying number of threads and number of episodes

Specify number of threads for evaluation using --num_processes and number of evaluation episodes per thread using --num_episodes. The Gibson val set consists of 14 scenes and 71 episodes per scene. Thus, we recommend using 14 threads for evaluation, and 71 episodes per thread.

For example, if you have 2 GPUs:

python  main.py --split val_mt --eval 1 \
--auto_gpu_config 0 -n 14 --num_episodes 71 --num_processes_per_gpu 7 \
--load_global pretrained_models/model_best.global --train_global 0 \
--load_local pretrained_models/model_best.local  --train_local 0 \
--load_slam pretrained_models/model_best.slam  --train_slam 0

Specifying map sizes

The full and local map sizes are specified using the following arguments:

--map_size_cm (default: 2400) 
--global_downscaling (default:2)

The default arguments lead to full map size of M = 480 (=24m) and local map size of G = 240 (=12m). Although the pre-trained models are trained with default map size for better training speed, they can be evaluated with larger full map size to improve performance. The full map size can be increased to M = 960 (=48m) using --map_size_cm 4800 --global_downscaling 4:

python  main.py --split val_mt --eval 1 \
--auto_gpu_config 0 -n 14 --num_episodes 71 --num_processes_per_gpu 7 \
--map_size_cm 4800 --global_downscaling 4 \
--load_global pretrained_models/model_best.global --train_global 0 \
--load_local pretrained_models/model_best.local  --train_local 0 \
--load_slam pretrained_models/model_best.slam  --train_slam 0

Evaluating on small and large scenes

To evaluate on small and large scenes on the gibson set separately, create small and large splits using:

python scripts/convert_datasets.py --split_by_size 1

This will create two datasets at data/datasets/pointnav/gibson/v1//val_mt_small/ (10 scenes) and data/datasets/pointnav/gibson/v1//val_mt_large/ (4 scenes). You can evaluate models on these datasets by changing --split and -n arguments in the above command to val_mt_small and 10 or val_mt_large and 4.

Visualization and printing images

For visualizing the agent observations and predicted map and pose, add -v 1 as an argument to the above command. This will require a display to be attached to the system.

To visualize on headless systems (without display), use --print_images 1 -d results/ --exp_name exp1. This will save the visualization images in results/dump/exp1/episodes/.

Both -v 1 and --print_images 1 can be used together to visualize and print images at the same time.

Troubleshooting

  • ImportError: cannot import name 'PILLOW_VERSION' from 'PIL'. New Pillow version breaks torchvision, see the issue here. Downgrading the pillow version seems to work:
pip install "pillow<7"

Tips

To silence habitat sim log add the following to your ~/.bashrc (Linux) or ~/.bash_profile (Mac)

export GLOG_minloglevel=2
export MAGNUM_LOG="quiet"