This is the official repository of the paper HIDRO-VQA
HIDRO-VQA: High Dynamic Range Oracle for Video Quality Assessment
Shreshth Saini*, Avinab Saha*, and Alan C. Bovik
3rd Workshop on Image/Video/Audio Quality in Computer Vision and Generative AI
WACV 2024
The code has been tested on Linux systems with Python 3.9. Please refer to requirements.txt for installing dependent packages.
In order to obtain quality-aware features or to start HDR quality-aware fine-tuning, checkpoints need to be downloaded. Download the checkpoint folder from the link and save them to the checkpoints folder.
For obtaining HIDRO-VQA features, the following command can be used. The features are saved in '.npy' format. It assumes the videos are stored in raw YUV 10-bit format, upscaled to 4K. Please change the path of the videos in line 97.
python demo_hidro_vqa_feats.py
Please take a look at the DATA folder for steps on how to prepare data for HIDRO-VQA pre-training.
Training with multiple GPUs using Distributed training
Run the following commands on different terminals concurrently. Please update the folder location of HDR Frames in the data_loader file located in modules/data_loader.py (Lines 119,137).
CUDA_VISIBLE_DEVICES=0 python train.py --nodes 3 --nr 0 --batch_size 256 --lr 0.1 --epochs 25
CUDA_VISIBLE_DEVICES=1 python train.py --nodes 3 --nr 1 --batch_size 256 --lr 0.1 --epochs 25
CUDA_VISIBLE_DEVICES=2 python train.py --nodes 3 --nr 2 --batch_size 256 --lr 0.1 --epochs 25
Note that in distributed training, batch_size value will be the number of images to be loaded on each GPU.
After HIDRO-VQA model pre-training, an SVR is trained using HIDRO_VQA features and corresponding ground truth quality scores from LIVE-HDR using the following command. It assumes features from each video is extracted (per frame) and stored using individual numpy files in a folder.
python train_svr.py --score_file <score_csv_file> --feature_folder <feature_folder_path> --train_and_test
Contributions are welcome! If you find any issues or have suggestions for improvements, please open an issue or submit a pull request.
@InProceedings{Saini_2024_WACV,
author = {Saini, Shreshth and Saha, Avinab and Bovik, Alan C.},
title = {HIDRO-VQA: High Dynamic Range Oracle for Video Quality Assessment},
booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops},
month = {January},
year = {2024},
pages = {469-479}}
This project is licensed under the MIT License.