MV-ROI: Multi-View ROI Prediction and Annotation

Installation

Setup the annotation framework environment

make setup

Read the bdda setup for how to setup the BDD-A model.

Dataset

Download the MV-ROI dataset.

• Use our dataset extractor to export image and label files from the dataset.
• Prepare the data for training the bdda model with the prepare module or directly use our finetuned weights in bdda/weights.

Annotation

General workflow of the annotation framework.

1. Data Generation: Use your own data and record from CARLA with our record module.
2. Data Format (Optional): Data require this format. Details for reindexing here.
3. Data Preparation: Prepare the data to be used by the bdda model with the prepare module.
4. Model Prediction: Generate ROI predictions for the data using the finetuned model.
5. Reformat Data: Reformat the data from bdda naming conventions to ours to be used in the further pipeline with the reformat module.
6. Pseudo Label Generation: Generate json labels from model predictions.
7. Create Consistent ROI Labels: Create consistent ROI json labels over all views.
8. Merge Samples: Merge all camera views of a sample
9. Human Inspection: Use the tool labelme to manually inspect and manipulate the labels
10. Split Samples: Split all camera views of a sample
11. Generate HDF5: Generate HDF5 files from labels and images

For details read the documentation of the Annotation Framework, BDDA Model and Record Module.

Paper

If you use MV-ROI please cite our paper.

Multi-View Region of Interest Prediction for Autonomous Driving Using Semi-Supervised Labeling, Markus Hofbauer, Christopher B. Kuhn, Jiaming Meng, Goran Petrovic, Eckehard Steinbach; ITSC 2020 [PDF]

@inproceedings{hofbauer_2020,
    title = {Multi-View Region of Interest Prediction for Autonomous Driving Using Semi-Supervised Labeling},
    booktitle = {23rd IEEE International Conference on Intelligent Transportation Systems (ITSC)},
    publisher = {IEEE},
    author = {Hofbauer, Markus and Kuhn, Christopher B. and Meng, Jiaming and Petrovic, Goran and Steinbach, Eckehard},
    address = {Rhodes, Greece}
    month = sep,
    year = {2020},
    pages = {1--6},
}

Development

We use pre-commit to manage our git pre-commit hooks. pre-commit is automatically installed from requirements-dev.txt. To set it up, call

git config --unset-all core.hooksPath  # may fail if you don't have any hooks set, but that's ok
pre-commit install --overwrite

Hooks Usage

With pre-commit, you don't use your linters/formatters directly anymore, but through pre-commit:

pre-commit run --file path/to/file1.cpp tools/second_file.py  # run on specific file(s)
pre-commit run --all-files  # run on all files tracked by git
pre-commit run --from-ref origin/master --to-ref HEAD  # run on all files changed on current branch, compared to master
pre-commit run <hook_id> --file <path_to_file>  # run specific hook on specific file