Datumaro with only annotations?

[knelson-hme]

Is it possible to use Datumaro with annotations only and no associated image files? I have a fairly large video file I am annotating in CVAT which I then export into Datumaro format without the "Save images" checked in CVAT (I do this because the resulting file exported from CVAT becomes very large if I do). I then want to do some post processing on the annotations, but I run into some issues when I try and run the following simple code:

data_path = "path/to/dataset"
dataset = Dataset.import_from(data_path)
for item in dataset:
    print(item)

It seems like the import_from method is working but when I try to loop through the dataset I get issues as Datumaro is trying to find an associated image file even though one doesn't exist. When I repeat the same steps above with "Save images" check in CVAT everything works, but this takes a lot of extra time and space to do as it creates a jpg of all my frames.

I have been able to loop through my annotations using the following dirty way, but I am sure there has to be a better way.

data_path = "path/to/dataset"
dataset = Dataset.import_from(data_path)
for item in dataset._data._source:
    print(item)

[vinnamkim]

Hi @knelson-hme,
I guess your problem happens here:

datumaro/src/datumaro/plugins/data_formats/cvat/base.py

Lines 367 to 376 in cce5fc9

	if osp.exists(image_path_opt_1):
	image = image_path_opt_1
	elif image_path_opt_2 and osp.exists(image_path_opt_2):
	image = image_path_opt_2
	elif "name" not in item_desc:
	# If --use-track flag is on
	# TODO: Revisit all the CVAT import/export parts.
	image = image_path_opt_1
	else:
	raise DatasetImportError(f"Cannot find an image which has name={name}.")

There is a file existence check during importing CVAT data format. It requires there exist actual image files in images_dir which is

datumaro/src/datumaro/plugins/data_formats/cvat/base.py

Lines 60 to 63 in cce5fc9

	rootpath = osp.dirname(path)
	images_dir = ""
	if osp.isdir(osp.join(rootpath, CvatPath.IMAGES_DIR)):
	images_dir = osp.join(rootpath, CvatPath.IMAGES_DIR)

To resolve this, you might create a soft directory link or copy-and-paste them to put your images in the right place (images_dir).

You can refer to the directory structure of CVAT data format here: https://openvinotoolkit.github.io/datumaro/latest/docs/data-formats/formats/cvat.html#import-cvat-dataset

Unfortunately, there is currently no functionality to assign images_dir outside. We will put this functionality on our backlog.

Thanks!

[knelson-hme]

Thank you for taking the time to read and reply to my post.

I just want to be clear that my dataset is in Datumaro format and not CVAT when I export it from the CVAT GUI interface.

When I run my code I get a problem with the following lines of code:

datumaro/src/datumaro/components/dataset_storage.py

Lines 259 to 264 in cce5fc9

	for i, item in enumerate(source):
	if item.media and not isinstance(item.media, media_type):
	raise MediaTypeError(
	"Unexpected media type of a dataset item '%s'. "
	"Expected '%s', actual '%s' " % (item.id, media_type, type(item.media))
	)

Where it hits the exception and says

datumaro.components.errors.MediaTypeError: Unexpected media type of a dataset item 'frame_000000'. Expected '<class 'datumaro.components.media.Image'>', actual '<class 'datumaro.components.media.PointCloudFromFile'>'

From my understanding this error is happening because I don't have any frame image data saved with my annotations. From your reply it seems like Datumaro requires image frame data to be located with the annotations.

Let me explain my original intention to hopefully clear up any confusion. I want to annotate a series of videos that are about 1 hour long each. I import the video into CVAT's GUI and do my annotations. In order to save space I was hoping I could just export the annotations in Datumaro format without also generating a bunch of individual jpeg frames which take up a lot of space. This could be the stupid step, but I then want to use my own python code to load the original video using opencv and also read in the Datumaro dataset annotations (that don't have any explicit image data saved with it) and do some post processing on annotations. My thinking was that I could use opencv to generate the individual frames from the original video file and that these should line up with the Datumaro annotations. I was thinking I could do it this way so I only had to have one copy of the original video without any extra jpeg frames saved to disk. Maybe this approach is flawed and I shouldn't be trying to do it this way. I am curious to know your thoughts.

[vinnamkim]

Hi You're welcome,

This seems a bug since I can see that the media type is PointCloud from your log,
datumaro.components.errors.MediaTypeError: Unexpected media type of a dataset item 'frame_000000'. Expected '<class 'datumaro.components.media.Image'>', actual '<class 'datumaro.components.media.PointCloudFromFile'>'.

Sorry for inconvenience, but long ago, CVAT forked out this repository and has been maintaining its own forked Datumaro. We might need to contact the CVAT team and recheck the video dataset exportation workflow at the same time.

Maybe this approach is flawed and I shouldn't be trying to do it this way. I am curious to know your thoughts.

I don't know what is your next workflow for the exported dataset exactly. However, for your question, my thought is that you would better to export your dataset with jpeg frames, if your next workflow requires multiple accesses for the video frames (e.g., model training). This is because you know video decoding costs enormously expensive computation.

[vinnamkim]

Root cause of this problem: cvat-ai/cvat#5924