PASCAL VOC XML is incorrect for OBB labels

[patel-zeel]

Describe the bug
PASCAL VOC XML export generates incorrect values for Oriented Bounding Boxes (OBB) labels.

To Reproduce

Expected behavior
I am not sure if PASCAL VOC XML supports OBB labels.

If it supports:
    It should correctly convert the labels.
else:
    It should first convert the labels to axis-aligned bounding boxes and then get the bounds (just like YOLO export).
    It may show a warning for this implicit conversion.

Screenshots

Labeling in label-studio

PASCAL VOC XML

<?xml version="1.0" encoding="utf-8"?>
<annotation>
<folder>images</folder>
<filename>2af821c3-28.2077.44.png</filename>
<source>
<database>MyDatabase</database>
<annotation>COCO2017</annotation>
<image>flickr</image>
<flickrid>NULL</flickrid>
<annotator>1</annotator>
</source>
<owner>
<flickrid>NULL</flickrid>
<name>Label Studio</name>
</owner>
<size>
<width>1120</width>
<height>1120</height>
<depth>3</depth>
</size>
<segmented>0</segmented>
<object>
<name>Airplane</name>
<pose>Unspecified</pose>
<truncated>0</truncated>
<difficult>0</difficult>
<bndbox>
<xmin>514</xmin>
<ymin>141</ymin>
<xmax>613</xmax>
<ymax>193</ymax>
</bndbox>
</object>
<object>
<name>Car</name>
<pose>Unspecified</pose>
<truncated>0</truncated>
<difficult>0</difficult>
<bndbox>
<xmin>819</xmin>
<ymin>317</ymin>
<xmax>918</xmax>
<ymax>379</ymax>
</bndbox>
</object>
</annotation>

Ploting PASCAL VOC and YOLO labels after exports

import matplotlib.pyplot as plt
img = plt.imread("images/2af821c3-28.2077.44.png")

fig, ax = plt.subplots(figsize=(8, 8))
plt.imshow(img)

# <xmin>514</xmin>
# <ymin>141</ymin>
# <xmax>613</xmax>
# <ymax>193</ymax>

x1 = 514
y1 = 141
x2 = 613
y2 = 193

plt.plot([x1, x2], [y1, y1], c='g')
plt.plot([x2, x2], [y1, y2], c='g')
plt.plot([x2, x1], [y2, y2], c='g')
plt.plot([x1, x1], [y1, y2], c='g')

# <xmin>819</xmin>
# <ymin>317</ymin>
# <xmax>918</xmax>
# <ymax>379</ymax>

x1 = 819
y1 = 317
x2 = 918
y2 = 379

plt.plot([x1, x2], [y1, y1], c='b')
plt.plot([x2, x2], [y1, y2], c='b')
plt.plot([x2, x1], [y2, y2], c='b')
plt.plot([x1, x1], [y1, y2], c='b', label='PASCAL VOC XML labels')

### YOLO
# 0 0.5042087542087542 0.14983164983164982 0.08922558922558928 0.047138047138047125
# 1 0.7449494949494948 0.33417508417508407 0.10281727886993011 0.10189583947751818
x1 = (0.5042087542087542 - 0.08922558922558928/2) * 1120
x2 = (0.5042087542087542 + 0.08922558922558928/2) * 1120
y1 = (0.14983164983164982 - 0.047138047138047125/2) * 1120
y2 = (0.14983164983164982 + 0.047138047138047125/2) * 1120

plt.plot([x1, x2], [y1, y1], c='g', linestyle='--', linewidth=5)
plt.plot([x2, x2], [y1, y2], c='g', linestyle='--', linewidth=5)
plt.plot([x2, x1], [y2, y2], c='g', linestyle='--', linewidth=5)
plt.plot([x1, x1], [y1, y2], c='g', linestyle='--', linewidth=5)

x1 = (0.7449494949494948 - 0.10281727886993011/2) * 1120
x2 = (0.7449494949494948 + 0.10281727886993011/2) * 1120
y1 = (0.33417508417508407 - 0.10189583947751818/2) * 1120
y2 = (0.33417508417508407 + 0.10189583947751818/2) * 1120

plt.plot([x1, x2], [y1, y1], c='b', linestyle='--', linewidth=5)
plt.plot([x2, x2], [y1, y2], c='b', linestyle='--', linewidth=5)
plt.plot([x2, x1], [y2, y2], c='b', linestyle='--', linewidth=5)
plt.plot([x1, x1], [y1, y2], c='b', label="YOLO labels", linestyle='--', linewidth=5)

plt.legend()
plt.savefig("tmp.png", dpi=200)

Environment (please complete the following information):

• OS: Ubuntu 20.04.6 LTS
• Label Studio Version 1.12.1

[luforestal]

Have you found a solution? I have a similar problem with my annotations. They look nice and well centered in LabelStudio but when I plot them in python, they look offset like 100 pixels.

[patel-zeel]

@luforestal Since even YOLO OBB export is not yet implemented (HumanSignal/label-studio-converter#281), we use the following workaround:

• Export to CSV format which has the following data:
  1. (x, y) - Anchor point (not the center but one of the corners)
  2. width and height
  3. rotation angle in degrees
  4. category
• Convert it to YOLO OBB format with the following custom script.

# @Author: Zeel B Patel, https://patel-zeel.github.io/
import numpy as np
import torch

from ultralytics.utils.ops import xywhr2xyxyxyxy
from typing import Union

label_map = {"Class A": 0, "Class B": 1}

def label_studio_to_yolo_obb(x1, y1, w, h, r, label) -> Union[np.ndarray, torch.Tensor]:
    """
    Convert from Label studio CSV (x1, y1, w, h, r) to YOLO OBB (x1, y1, x2, y2, x3, y3, x4, y4) format
    
    x1: x-cordinate of one corner of the box, range(0, 100)
    y1: y-cordinate of one corner of the box, range(0, 100)
    w: width of the box, range(0, 100)
    h: height of the box, range(0, 100)
    r: rotation angle of the box in degrees
    label: label of the box
    
    Returns: Label in YOLO OBB format -> array([label_id, x1, y1, x2, y2, x3, y3, x4, y4])
    """
    
    if isinstance(x1, torch.Tensor):
        handle = torch
        handle.concatenate = torch.cat
        get_array = lambda x: torch.tensor(x)
    else:
        handle = np
        get_array = lambda x: np.array(x)
        
    r = handle.radians(r)
    
    cos_rot = handle.cos(r)
    sin_rot = handle.sin(r)
    
    x_c = x1 + w / 2 * cos_rot - h / 2 * sin_rot
    y_c = y1 + w / 2 * sin_rot + h / 2 * cos_rot
    
    xywhr = get_array([x_c, y_c, w, h, r])
    xyxyxyxy = xywhr2xyxyxyxy(xywhr).ravel()
    
    # Normalize to range(0, 1)
    xyxyxyxy = xyxyxyxy / 100
    
    label_id = get_array([label_map[label]])
    yolo_label = handle.concatenate([label_id, xyxyxyxy])
    return yolo_label

• Once we get x1, y1, x2, y2, x3, y3, x4, y4, you can get the bounds and put it inside the XML programatically

xmax = max([x1, x2, x3, x4])
xmin = min([x1, x2, x3, x4])
ymax = max([y1, y2, y3, y4])
ymin = min([y1, y2, y3, y4])

[sajarin]

Thanks for sharing your workaround @patel-zeel, we will update when YOLO OBB export is finally implemented. Otherwise, closing this issue for now!