SEA-RAFT

This repository contains the source code for our paper SEA-RAFT, which is a fast, simple, and effective framework for optical flow estimation. On Spring benchmark, SEA-RAFT can process 1080p image pairs at >20FPS while achieving state-of-the-art performance.

If you find SEA-RAFT useful for your work, please consider citing our academic paper:

SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow

Yihan Wang, Lahav Lipson, Jia Deng

@misc{wang2024searaft,
      title={SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow}, 
      author={Yihan Wang and Lahav Lipson and Jia Deng},
      year={2024},
      eprint={2405.14793},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Requirements

Our code is developed with pytorch 2.2.0, CUDA 12.2 and python 3.10.

conda create --name SEA-RAFT python=3.10.13
conda activate SEA-RAFT
pip install -r requirements.txt

Model Zoo

Please download the models from google drive and put them into the models folder.

Required Data

To evaluate/train SEA-RAFT, you will need to download the required datasets.

• FlyingChairs
• FlyingThings3D
• Sintel
• KITTI
• HD1K
• TartanAir
• Spring

By default datasets.py will search for the datasets in these locations. You can create symbolic links to wherever the datasets were downloaded in the datasets folder. Please check RAFT for more details.

├── datasets
    ├── Sintel
    ├── KITTI
    ├── FlyingChairs/FlyingChairs_release
    ├── FlyingThings3D
    ├── HD1K
    ├── spring
        ├── test
        ├── train
        ├── val
    ├── tartanair

Training, Evaluation, and Submission

Please refer to scripts/train.sh, scripts/eval.sh, and scripts/submission.sh for more details.

Acknowledgements

This project would not have been possible without relying on some awesome repos: RAFT, unimatch, Flowformer, ptlflow, and LoFTR. We thank the original authors for their excellent work.