Segment 2D radiographs (e.g. abdominal x-rays) using MONAILabel and Slicer?

[mikebind]

I am an experienced 3D Slicer user and have been exploring MONAILabel use for CT segmentation. So far, I am impressed with the relative ease of setting up MONAILabel and getting to an iteratively improvable segmentation model which interacts well with Slicer. I am also interested in exploring the use of MONAILabel and Slicer for 2D images. I realize that Slicer is primarily oriented towards 3D images, but it handles 2D images just fine and it is an environment I know well. I also see that there is a Pathology app in addition to the Radiology app, and I assume the Pathology app is set up to operate on 2D images (please correct me if I am wrong about this). So I have two related questions: Can the pathology app be easily modified to operate on 2D radiographs rather than slide images? (I assume the answer to this is yes, but am not sure if there are issues I should be aware of). Second, can the pathology app be used from Slicer? And are there any issues to be aware of in that respect? Nearly all the documentation I can find relates to the Radiology app, so I am not as clear on how the Pathology app works.

[jlvahldiek]

I can confirm that MONAILabel with 3D Slicer can be used for segmentation of 2D X-ray images.

My team uses single DICOM files for the images and .seg.nrrd files for the annotations to allow overlapping segmentations.

We have completed several segmentation tasks with X-ray images. Each time, we proceeded like this:

1. Several hundred images were manually segmented from scratch by several different human annotators using the MONAILabel extension in 3D slicer.
2. Training of a segmentation model with MONAI (independent of MONAILabel).
3. Then use this model in MONAILabel for pre-segmentations and for active learning strategies (epistemic). This saved a lot of time for the human annotators for further segmentations.
4. Final model training.

[mikebind]

Thank you for your response! Is there a reason you separate step 2 from MONAILabel? In step 3, are you using the "Submit Label" button to save final segmentations, or are you saving them separately because something would not work correctly using the MONAILabel machinery?

[SachidanandAlle]

you can define the training task in monailabel.. it's possibly much easier to define few simple pre/post transforms etc.. and get the job done.. it basically abstracts few details of monai.. this is how we pretrained our models.. but if you are interested to customize a lot and want to dig more deeper.. depending your use-case on training, you can use monai examples..

[jlvahldiek]

At that time we were exploring a lot and tried different transforms and model archs that was the only reason for the separation. As @SachidanandAlle noticed when you know your transforms and archs and have a stable training pipeline training in MONAILabel works very similar to Monai core.

Of coarse we are using the submit button to save labels. This way it is much easier for the annotators to save their work instead of saving directly to the file system.

We are running MONAILabel for several months now with different tasks and more than 20 different human annotators. It was a great experience to switch to MONAILabel. Also our annotators like the workflow in Slicer.

[SachidanandAlle]

As @jlvahldiek mentioned you can continue using the radiology app to modify and support your 2D models. Client is secondary.
Pathology app is mainly for WSI kind of images.. it will work for 2D.. but you may don't get confused with the different terminology used.

You can see deepgrow 2D model.. (however it still works on 3D but takes care of slicing it and running infer on single slice).. this kind may not be required if you have a viewer which takes care of loading 2D images and updating the masks accordingly.

[SenhorUnk]

Hi there,

I just found this thread, which goes in the direction of my problem.

I would like to make use of the interactive segmentation functionalities in 3D Slicer for annotating 2D-images and improving the segmentation model over time to reduce the annotation effort.

I am running the MONAI Label radiology sample app with deepgrow_2d selected when starting the server. The input are some random 2D-images (I am aware that the pretrained deepgrow_2d model will bring any benefit with these images).

When I place a foreground or background point from the "SmartEdit / Deepgrow"-tab, I am getting the error as shown in the picture below. The error seems to be related to the shape of the input image. But unfortunately, I don't know how to solve the problem.
I would be happy about any help - thanks!

The control "Show Details" gives the following error:

The log in the terminal looks like this:

[SachidanandAlle]

Even though it's deepgrow 2d.. the current version/model needs 3D (volume) input.. heavily trained on CT images.. it won't work on PNG with 3 channel..
If you are looking for pure 2D models.. then I suggest to have a look on endoscopy/pathology apps..

[SenhorUnk]

Thanks for the quick response.

Could you explain the difference between the 2D and 3D deepgrow, when the input still needs to be 3D?
@jlvahldiek said, his team segmented 2D x-ray images using 3D-slicer and probably an own model, which is capable of 2D-image-input. This is also a potential solution for my use case.

[SachidanandAlle]

You can check the pre transforms.. u need to fix yours if u want to support 2d images as input