FIR-based Future Trajectory Prediction in Nighttime Autonomous Driving

The performance of the current collision avoidance systems in Autonomous Vehicles (AV) and Advanced Driver Assistance Systems (ADAS) can be drastically affected by low light and adverse weather conditions. Collisions with large animals such as deer in low light cause significant cost and damage every year. In this project, we propose the first AI-based method for future trajectory prediction of large animals and mitigating the risk of collision with them in low light.

In order to minimize false collision warnings, in our multi-step framework, first, the large animal is accurately detected and a preliminary risk level is predicted for it and low-risk animals are discarded. In the next stage a multi-stream CONV-LSTM-based encoder-decoder framework is designed to predict the future trajectory of the potentially high-risk animals. The proposed model uses camera motion prediction as well as the local and global context of the scene to generate accurate predictions. Furthermore, this project introduces a new dataset of FIR videos for large animal detection and risk estimation in real nighttime driving scenarios. Our experiments show promising results of the proposed framework in adverse conditions. This repository contains the main blocks of our code.

Paper:

FIR-based Future Trajectory Prediction in Nighttime Autonomous Driving

IEEE Intelligent Vehicles 2023 (IEEE IV 2023)

citation:

@article{rahimpour2023fir,
  title={FIR-based Future Trajectory Prediction in Nighttime Autonomous Driving},
  author={Rahimpour, Alireza and Fallahinia, Navid and Upadhyay, Devesh and Miller, Justin},
  journal={arXiv preprint arXiv:2304.05345},
  year={2023}
}

=======

Code:

Please refer to this repo for the code and more details: NightVision code and details