models/mobile-sam/

[giscus[bot]]

models/mobile-sam/

Learn more about MobileSAM, its implementation, comparison with the original SAM, and how to download and test it in the Ultralytics framework. Improve your mobile applications today.

https://docs.ultralytics.com/models/mobile-sam/

[AnandSingh-0619]

Hey!
How can I run batch inference with multiple images and bounding boxes on the Mobile SAM model?

[glenn-jocher]

Hey there! 👋

Running batch inference with multiple images and bounding boxes on MobileSAM is quite straightforward. Here’s a quick example of how you could do that:

from ultralytics import SAM

# Load the MobileSAM model
model = SAM('mobile_sam.pt')

# List of images and their corresponding boxes
images = ['path/to/image1.jpg', 'path/to/image2.jpg']
boxes = [[439, 437, 524, 709], [500, 400, 550, 720]]  # Example boxes for each image

# Predict segments based on box prompts for each image
for image, box in zip(images, boxes):
    result = model.predict(image, bboxes=[box])
    # You can now process the result as needed

This code loads the model, iterates over your images and boxes, and then performs prediction for each image using the specified boxes. Don't forget to adjust the image paths and box coordinates according to your data. 😊

Let us know if you need more help!

[AnandSingh-0619]

I'm working with the Ultralytics Mobile SAM model and need to perform batch inference for multiple images, each with its own set of bounding boxes, without using a for loop to iterate over each image. How can I efficiently run batch inference on the Mobile SAM model with this setup?

[glenn-jocher]

Hi there! For batch inference with the Mobile SAM model without iterating over each image using a for-loop, you can leverage the batch processing capabilities of PyTorch. While the Mobile SAM specifically might not showcase an example for this, the concept applies broadly across models. Here’s a brief idea on how you might approach it:

1. Stack your images into a single tensor. Ensure all images are of the same size or appropriately padded.
2. Organize your bounding boxes in a similar batched manner.
3. Pass the batched images and bounding boxes to your model’s predict function.

Here’s a simplified example:

import torch
from ultralytics import SAM

# Assume images is a list of your pre-processed images tensors
# and bboxes is a list of corresponding bounding boxes
images_tensor = torch.stack(images)  # shape [N, C, H, W]
bboxes_tensor = torch.tensor(bboxes)  # shape [N, 4] for [x1, y1, x2, y2] format

model = SAM('mobile_sam.pt')
results = model.predict(images=images_tensor, bboxes=bboxes_tensor)

Remember, images need to be tensors and all preprocessing like normalization should align with MobileSAM's requirements. The key here is to properly manage your data’s shape. 😊

Hope this helps! If you have more specific needs or run into any issues, feel free to ask.

[FlySka]

Hi,
Do you have any notebook or repository for fine-tuning mobileSAM? Could it be done in the same way as with the regular SAM?

[pderrenger]

Hi there,

Yes, fine-tuning MobileSAM can generally be approached in the same manner as with the regular SAM. Since MobileSAM retains the same pipeline, including pre-processing, post-processing, and interfaces, adaptations for MobileSAM from SAM should be minimal.

Here's a quick Python snippet that illustrates how you could load and use MobileSAM for predictions, which mirrors the usage for SAM:

from ultralytics import SAM

# Load the MobileSAM model
model = SAM('mobile_sam.pt')

# Predict a segment based on a point prompt
result = model.predict('path/to/your/image.jpg', points=[x, y], labels=[label_id])

You can apply any fine-tuning procedures used for SAM in a similar fashion for MobileSAM. If you need more detailed steps or specific adjustments for fine-tuning, feel free to ask!

Best regards!

[Kunal8020]

How can I use this for multi class segmentation. I am detecting the multiple objects using YOLO-World. I want to use the bounding box results from YOLO-World as prompt to MobileSAM. How can I use to segment different objects in the same image??

[pderrenger]

Hello!

To use MobileSAM for segmenting multiple objects in the same image using bounding box results from YOLO-World, you can follow these steps:

1. Run YOLO-World Detection: First, use YOLO-World to detect objects in your image. This will provide you with bounding boxes for each detected object.

2. Prepare Bounding Box Inputs: Extract the bounding box coordinates from the YOLO-World output. These coordinates will be used as prompts for MobileSAM.

3. Segment with MobileSAM: For each bounding box, use the MobileSAM model to segment the object within that box. You can do this by passing the bounding box coordinates to MobileSAM's predict function, similar to the box prompt example:

   from ultralytics import SAM
   
   # Load the model
   model = SAM("mobile_sam.pt")
   
   # Predict a segment based on a box prompt
   model.predict("path/to/image.jpg", bboxes=[x1, y1, x2, y2])

   Repeat this for each bounding box provided by YOLO-World.

This approach allows you to segment each detected object individually. If you need further assistance or have more specific requirements, feel free to ask!

[gopalkatushar]

This is great. Is there a way to segment everything like we have in fastsam, without prompting?

[glenn-jocher]

@gopalkatushar mobileSAM currently requires prompts for segmentation, similar to the original SAM. For more details, please refer to the MobileSAM documentation.

[ismailwale]

How can I make an inference of a video using frame by frame in loop using Mobilesam ?because I tried multiple times making inference directly with video predictor I constantly running out of memory despite setting stream =True since SAM is trying to keep the memory of 2000 frames of the video at ones . I will appreciate any ideal on how to predict and segment the video as a single frame image at every instance and put everything in a loop.

[glenn-jocher]

@ismailwale to process video frame-by-frame with MobileSAM and avoid memory overflow, load the video using OpenCV, iterate through each frame in a loop, perform inference using MobileSAM on individual frames, and release unused memory after each iteration. Make sure not to enable streaming mode. Here is a basic example structure:

import cv2
from ultralytics import SAM

# Initialize the VideoCapture and load the model
cap = cv2.VideoCapture("path/to/video.mp4")
model = SAM("mobile_sam.pt")

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    # Perform inference on the current frame
    results = model.predict(frame)

    # Process results or save outputs
    # ...

cap.release()  # Release the video

This approach minimizes memory usage by processing frames sequentially. Let us know if you need further clarification!

[ismailwale]

Thanks so much , your lines of code really jump-start my thinking. I sincerely appreciate. Please keep on with your good work . Be blessed

[glenn-jocher]

You're very welcome! I'm glad the code helped you get started. Credit goes to the amazing YOLO community and the Ultralytics team for developing these fantastic tools. Best of luck with your project, and let us know if you have more questions!

[tunahsu]

Hi!
Is it possible to filter out low-quality masks during inference using confidence/score?

[glenn-jocher]

@tunahsu yes, you can filter out low-quality masks during inference by adjusting the confidence threshold using the conf parameter when running the model. For more details on setting this parameter, you can refer to the MobileSAM documentation. Let us know if you have further questions!