Skip to content

CholecTriplet 2022 challenge on surgical action triplet detection

License

Notifications You must be signed in to change notification settings

CAMMA-public/cholectriplet2022

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

33 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation


CholecTriplet2022

CholecTriplet 2022 challenge on surgical action triplet detection

C.I. Nwoye, A. Murali, S. Sharma, T. Yu, K. Yuan, D. Alapatt, A. Vardazayan, and N. Padoy

t50-logo-2022

This repository contains some implementation code: mock demo model, data loader, docker build guides, self-validator system, and evaluation scripts.


Getting started

All information about the challenge can be found on the Grand-challenge website.


Data loader

© Cholect45 Git Repo

Practice on sample data

Easy starting code of simple model in TensorFlow or PyTorch on small sample data from CholecT50.


Evaluation metrics

  pip install ivtmetrics

or

 conda install -c nwoye ivtmetrics

Docker and validation guide

  • Detailed instruction to build, test, validate, and run your method Docker on the challenge Dockerhub in provided here

Submission protocol

  • Follow the guide to upload your final Docker image.
  • Visit the submission page to submit your challenge reports


Organizer's Baseline Method

The Rendezvous (RDV) for surgical action triplet recognition is modified to produce bounding boxes for the instrument tip of every recognized triplet.

This is made possible by the weakly supervised localization (WSL) aspect of the RDV. The WSL branch in RDV's encoder is responsible for instrument detection. It is trained on instrument's binary presence labels only. The localization is done via the resulting class activation map (CAM). We added an adhoc function to extract the bounding box coordinates for every positive activation in this layer.

The classifier in the RDV's decoder produces a vector of probility scores for the triplet class prediction. We extend this with a heuristic function to associate every extracted instrument bounding boxes to the corresponding triplet instance within a given frame.

Model produces both vector of classwise probability scores for triplet recognition + bounding boxes of the instruments paired to positive triplet classes. The output can be formatted as a list of list (lol) or list of dict (lod) for each frame containing several instance triplet-box predictions. Full video prediction can be saved as a .TXT file or as a .JSON file

Rendezvous_Det code and weights is provided here.

Requirements:

  • PIL
  • Python >= 3.5
  • Pyorch >= 1.10.1
  • TorchVision >= 0.11
  • ivtmetrics

To Run

You need to map input/ and output/ directories to the corresponding input/ and output/ directories in the organizer's host folder.

    cd ..
    cd rendezvous_det
    python3 main.py \
		--input_dir=/path/to/organizer's/input/data \
		--output_dir=/path/to/organizer's/output/results \
		--gpu='0'

Results are saved as .JSON in the output folder.

During Docker run, the input_dir and output_dir should be default as in the argparse field.

To evaluate performance

To see the model performance, execute thr following commands:

    cd ..
    cd $HOST/validator
    python3 eval.py --gt_dir=/path/to/groundtruth --pd_dir=/path/to/predictions --log_dir=/path/to/write/results --threshold 0.5

Participants Code

Participants can release their code on their own discretion. All released code would appear here:

Method Team Institution Repository
1. RDV-Det CAMMA University of Strasbourg, France GitHub
2. AtomTKD SHUANGCHUN Southern University of Science and Technology, China Link unavailable
3. DATUM KLIV-IITKGP Indian Institute of Technology Kharagpur, India Link unavailable
4. Distilled-Swin SDS-HD German Cancer Research Center (DKFZ), Germany Link unavailable
5. DualMFFNet SK Muroran Institute of Technology, SK, Japan Link unavailable
6. EnoSurgTRD INTUITIVE-CORTEX-ML Intuitive Surgical, USA Link unavailable
7. IF-Net CAMP Technical University Munich, Germany Link unavailable
8. MTTT CITI Shanghai Jiao Tong University, China Link unavailable
9. ResNet-CAM-YOLOv5 WINTEGRAL Wintegral GmbH, Germany Link unavailable
10. SurgNet 2AI-ICVS Applied Artificial Intelligence Laboratory, Portugal Link unavailable
11. URN-Net URN University College London, UK Link unavailable

Maintainer @ camma 2022

About

CholecTriplet 2022 challenge on surgical action triplet detection

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published