[MQTT] Any plans to update the reviews topic to associate labels with event IDs?

[darnmason]

I'm using the reviews topic for HA notifications. The relevant data (copied from the docs):

{
  "detections": [
    "1718987128.947436-g92ztx",
    "1718987148.879516-d7oq7r",
    "1718987126.934663-q5ywpt"
  ],
  "objects": ["person", "car"],
  "sub_labels": ["Bob"],
  "zones": ["front_yard"],
  "audio": []
}

I'm currently plucking the first event ID from the detections array to get an image for a notification. But I'd like to prioritise an image of a person over a car, for example. With the current structure that is not possible, well without making API calls to get more information about each event.

Would it be a relatively simple change to expand the schema to include these associations?
Something like the following would be backwards-compatible:

{
  "detections": [
    "1718987128.947436-g92ztx",
    "1718987148.879516-d7oq7r",
    "1718987126.934663-q5ywpt"
  ],
  "objects": ["person", "car"],
  "sub_labels": ["Bob"],
  "zones": ["front_yard"],
  "audio": [],
  "events": [{
    "id": "1718987128.947436-g92ztx",
    "object": "person",
    "sub_labels": ["Bob"],
    "zones": ["front_yard"],
    "audio": []
  }, {
    "id": "1718987128.947436-d7oq7r",
    "object": "car",
    "sub_labels": [],
    "zones": [],
    "audio": []
  }, {
    "id": "1718987128.947436-q5ywpt",
    "object": "car",
    "sub_labels": [],
    "zones": [],
    "audio": []
  }]
}

I'm not immediately sure how easy this structure would be to work with in HA templates but just an example that provides the info I'd like.

[NickM-27]

No, we do not want to keep track of every object that was part of the review sequence and all of its attributes. Review items designate a period of activity not an entire set of objects. For busy periods of time this payload could become very large with a lot of redundant information.

It should be possible for the review thumbnail to be made available in the Frigate HA integration, the preview gif / video already is available in the integrations notifications api and is often more suitable for understanding a notification than a still image.

[darnmason]

Thanks for the quick reply. I tested out both gif and video clips in notifications but found myself going back to simple thumbnails. I guess I'm mostly interested in people so I want to know at a glance who the subject of the review is, and then decide if it's worth my time reviewing the footage. 90% of the time it's not as I recognise and trust the person.

But yeah sometimes I get a thumb of a car when the review also includes a person, I assume depending on which one was detected first. Does the review thumbnail have logic to prioritise which object to show?

The review data includes the thumb_url: /media/frigate/clips/review/thumb-<camera>-<review-id>.webp
Is the intention that this value be used as a media_source in HA?

[NickM-27]

The review thumbnails prioritize the moment that the highest number of objects were visible as this gives the most possible context.