Skip to content
/ HSTforU Public

This is an official implement for "HSTforU: Anomaly Detection in Aerial and Ground-based Videos with Hierarchical Spatio-Temporal Transformer for U-net"

Notifications You must be signed in to change notification settings

vt-le/HSTforU

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

34 Commits
 
 
 
 
 
 
 
 

Repository files navigation

HSTforU: Anomaly Detection in Aerial and Ground-based Videos with Hierarchical Spatio-Temporal Transformer for U-net

This is the code for HSTforU: Anomaly Detection in Aerial and Ground-based Videos with Hierarchical Spatio-Temporal Transformer for U-net Hits.

Related Works

MoGuP: See MoGuP:Motion-guided Prediction for Video Anomaly Detection.

ASTNet: See Attention-based Residual Autoencoder for Video Anomaly Detection.

Updates

  • [8/19/2023] Code of HSTforU is released (with password). We will remove the password after this paper is accepted.

Setup

The code can be run under any environment with Python 3.7 and above. (It may run with lower versions, but we have not tested it).

Install the required packages:

pip install -r requirements.txt

Clone this repo:

git clone https://github.com/vt-le/HSTforU.git
cd HSTforU/

We evaluate HSTforU on:

Dataset Link
UCSD Ped2 Google drive
CUHK Avenue Google drive
ShanghaiTech Google drive
Drone-Anomaly Google drive

A dataset is a directory with the following structure:

$ tree data
ped2/avenue
├── training
│   └── frames
│       ├── ${video_1}$
│       │   ├── 000.jpg
│       │   ├── 001.jpg
│       │   └── ...
│       ├── ${video_2}$
│       │   ├── 00.jpg
│       │   └── ...
│       └── ...
├── testing
│   └── frames
│       ├── ${video_1}$
│       │   ├── 000.jpg
│       │   ├── 001.jpg
│       │   └── ...
│       ├── ${video_2}$
│       │   ├── 000.jpg
│       │   └── ...
│       └── ...
└── ped2/avenue.mat

shanghaitech
├── training
│   └── frames
│       ├── ${video_1}$
│       │   ├── 000.jpg
│       │   ├── 001.jpg
│       │   └── ...
│       ├── ${video_2}$
│       │   ├── 00.jpg
│       │   └── ...
│       └── ...
├── testing
│   └── frames
│       ├── ${video_1}$
│       │   ├── 000.jpg
│       │   ├── 001.jpg
│       │   └── ...
│       ├── ${video_2}$
│       │   ├── 000.jpg
│       │   └── ...
│       └── ...
└── test_frame_mask
    ├── 01_0014.npy
    ├── 01_0015.npy
    └── ...

drone
├──bike
│  ├──training
│  │  └── frames
│  │      ├── ${video_1}$
│  │      │   ├── 0.jpg
│  │      │   ├── 1.jpg
│  │      │   └── ...
│  │      ├── ${video_2}$
│  │      │   ├── 00.jpg
│  │      │   └── ...
│  │      └── ...
│  ├── testing
│  │   └── frames
│  │       ├── ${video_1}$
│  │       │   ├── 000.jpg
│  │       │   ├── 001.jpg
│  │       │   └── ...
│  │       ├── ${video_2}$
│  │       │   ├── 000.jpg
│  │       │   └── ...
│  │       └── ...
│  └── annotation
│      ├── 01.npy
│      ├── 0.npy
│      └── ...
├── highway
│   ├── ...
└── ...

Evaluation

Please first download the pre-trained model

Dataset Pretrained Model
UCSD Ped2 Google drive
CUHK Avenue Google drive
ShanghaiTech Google drive
Drone-Anomaly Google drive

To evaluate a pretrained HSTforU on a dataset, run:

 python test.py \
    --cfg <path/to/config/file> \
    --pretrained </path/to/pre-trained/model> \
    [--batch-size <batch-size> --tag <job-tag>]

For example, to evaluate HSTforU on Ped2:

python test.py \
    --cfg config/scripts/ped2/ped2_pvt2_hst.yaml \
    --model-file output/HSTforU/ped2_pvt2_hst/ckpt_ped2.pth

Configuration

  • We use YAML for configuration.
  • We provide a couple preset configurations.
  • Please refer to config.py for documentation on what each configuration does.

Citing

If you find our work useful, please consider citing:

@article{hstforu,
title = {HSTforU: Anomaly Detection in Aerial and Ground-based Videos with Hierarchical Spatio-Temporal Transformer for U-net},
author = {Le, Viet-Tuan and Jin, Hulin and Kim, Yong-Guk},
}

Contact

For any question, please file an issue or contact:

Viet-Tuan Le: vt-le@outlook.com