🎓 DLMA: Deep Learning-Enabled Morphmetric Analysis

DLMA workflow: Aiming fast toxicology screening using zebrafish based on Deep learning (Use Detectron2 framework)

👏 Abstract

This AI project is a detection task for identifying zebrafish organs and phenotypes in micrographs, which is based on the Meta AI project, Detectron2 version 0.4.1. It mainly used Mask R-CNN for training and validating. It has 16 detected objects, including 8 specific organs and 8 specific abnormal phenotypes.

Inference results by Mask R-CNN

🐰 How to start

To use our zebrafish AI detection, you have to learn how to install and utilize the detectron2, please refer the detectron2 instructions.

Firstly, install python3.8 or higher version, detectron2 (0.4.1) and other dependent libraries (see zebrafish_maskrcnn.py). Among all the libraries, fishutil and fishclass are two customized python libraries for transforming the model outputs to quantitative parameters of zebrafish.

Then put the pre-trained model weights in your specified path.

🐾 Download the image library and pre-trained model weights

mask r-cnn & resnet_r_101/tensormask & resnet_r_50: download here

test_image:/train_image: download here test train

# Modify some necessary paths
# path to model weights
cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth")
...
# path to images that pending to detect
path = 'your/specified/path'
...
# save your results
df.to_csv(os.path.join('.../results/', 'results.csv'))

Finally, run the inference.py file in inference mode locally.

# get help
# specify the detect images and image type as well as the output folder
python inference.py --help

If you want to use TensorMask to identify the objects, please refer to another python file, zebrafish_tensormask.py. Through this model you can also see the identification results in real-time, which is supported by Open CV.

We have opened our image library and annotations, if you want to re-train this model locally, please download the train.json, test.json, train images and test images(see images_url.txt). The json file contains annotations of COCO style. After downloading the images and files, register the coco instances in your code, modify the name and path of the register.

# train registry
from detectron2.data.datasets import register_coco_instances
register_coco_instances("zebrafish_train", {}, "/your/path/train.json",
                        "/your/path/train_images/")
register_coco_instances("zebrafish_train", {}, "/your/path/test.json",
                        "/your/path/test_images/")

🐣 Expected results

After the model inference, we can acquire a csv file which cotains the quantitative parameters of specific organs and abnormal phenotypes, these information demonstrate every detail of one specific zebrafish, and we can use them to analyze the developmental status of each zebrafish.

Developer comments

We will continually update this repository and add more images to our library, if you want to use this work, please cite our research article. Citation infos:{ Deep Learning-Enabled Morphometric Analysis for Toxicity Screening Using Zebrafish Larvae Gongqing Dong, Nan Wang, Ting Xu, Jingyu Liang, Ruxia Qiao, Daqiang Yin, and Sijie Lin Environmental Science & Technology 2023 57 (46), 18127-18138 DOI: 10.1021/acs.est.3c00593 }