138 Image level tags #277
138 Image level tags #277
Conversation
lessej
added
In Progress
|
@nathanielrindlaub This is the api PR for the image tags. Should be ready for review now! |
| "$pull": { | ||
| "tags": input.tagId | ||
| } | ||
| }); |
There was a problem hiding this comment.
Have you stress tested this yet? How many tags can we remove on images within the 30 second time out? I see we capped it at 500 images but is that just arbitrary? I'm guessing that because it's a single updateMany call it is probably pretty efficient, but we should understand what the limit is. We have some projects with ~2 million images so you can imagine at that volume we might run into timeouts.
There was a problem hiding this comment.
Hey @nathanielrindlaub sorry I forgot to get back to fixing this. I put up a bug fix because it seems the deletion wasn't working as I expected. I did some quick checks of performance with as many images as possible in the current project. Please see below
| Type | Image Count | Duration |
|---|---|---|
| Cold Start | 1014 | 4118.89ms |
| Cold Start | 1014 | 4150.08ms |
| Cold Start | 1014 | 4233.56ms |
| Cold Start | 1014 | 4402.07ms |
| Cold Start | 1014 | 4036.09ms |
| Cold Start | 1014 | 3992.30ms |
| Cold Start | 1 | 3900.83ms |
| Cold Start | 1 | 3837.03ms |
| Cold Start | 1 | 4230.82ms |
| Cold Start | 1 | 3829.27ms |
| Cold Start | 1 | 3930.13ms |
| Cold Start | 1 | 4083.47ms |
| Warm | 1014 | 805.65ms |
| Warm | 1014 | 886.84ms |
| Warm | 1014 | 819.05ms |
| Warm | 1014 | 786.62ms |
| Warm | 1014 | 781.23ms |
| Warm | 1014 | 870.32ms |
| Warm | 1 | 653.31ms |
| Warm | 1 | 643.08ms |
| Warm | 1 | 634.07ms |
| Warm | 1 | 622.23ms |
| Warm | 1 | 619.41ms |
| Warm | 1 | 701.56ms |
These were taken on the local serverless offline lambda so I don't know if they will be useful. I used the shell in Compass to add a tag to all images in the default project and then removed them all using the UI which calls the local API.
I believe the 500 image limit is a limit for removing images with a specific label (object label). Tags is uncapped at the moment.
This data is likely not conclusive by any means so if you have a better way to stress test it closer to 2 million images at once, please let me know!
There was a problem hiding this comment.
I believe the 500 image limit is a limit for removing images with a specific label (object label). Tags is uncapped at the moment.
@JesseLeung97, sorry I'm not 100% tracking what you mean here. Mind elaborating a bit?
There was a problem hiding this comment.
@nathanielrindlaub Sorry for the confusion. I was responding to this
How many tags can we remove on images within the 30 second time out? I see we capped it at 500 images but is that just arbitrary?
This 500 image removal limit is not for this feature (image level tags). I believe it's for object labels.
There was a problem hiding this comment.
@JesseLeung97 oh sorry I was confused by the MAX_TAG_DELETE variable in this line, but I just searched and it doesn't seem to be used anywhere.
There was a problem hiding this comment.
@nathanielrindlaub In testing on dev with 52k images:
| Image Count | Duration |
|---|---|
| 51889 | 10.878s |
| 51889 | 10.973s |
| 51889 | 10.548s |
| 51889 | 11.230s |
| 51889 | 11.120s |
I did not observe any timeouts when removing the tag from 52k images and the max runtime was just under 12s. I arbitrarily set the upper limit for deleting at 50,000 given that's a round number near the max that we've been able to test. If we can add more images, I'm happy to continue raising the limit until it breaks.
Context
Foundational implementation for tnc-ca-geo/animl-frontend#138
Related PR
animl frontend