Using EfficientDet in the camera

[BenJam29]

Hi everyone,

I was developing a computer vision pipeline on Axis camera model Q1656-LE Box. I installed Axis ACAP and Axis computer vision SDK using Docker and everything is functional when I use the default detection model which is SSD Mobilenet V2. (Screenshot below).

This detection model is configured in the following files as far as I learned.

• app/object-detector-python/Dockerfile.model

`ARG ARCH=aarch64

FROM arm64v8/alpine as model-image-aarch64

FROM model-image-${ARCH}

# Get SSD Mobilenet V2
ADD https://github.com/google-coral/edgetpu/raw/master/test_data/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite models/
ADD https://github.com/google-coral/edgetpu/raw/master/test_data/ssd_mobilenet_v2_coco_quant_postprocess.tflite models/
ADD https://github.com/google-coral/edgetpu/raw/master/test_data/coco_labels.txt models/
ADD https://github.com/google-coral/edgetpu/raw/master/test_data/COPYRIGHT models/

CMD /bin/ls /models`

• config/env.aarch64.artpec8

`MODEL_PATH=/models/ssd_mobilenet_v2_coco_quant_postprocess.tflite
INFERENCE_SERVER_IMAGE=axisecp/acap-runtime:0.6-aarch64
INFERENCE_SERVER_COMMAND=/usr/bin/acap_runtime -p 8501 -j 12 `

What I need to do is to replace this default detection model with other open-source model like EfficientDet Lite0 (from coral AI model zoo here) or any other newly created model. I tried to change the two files above to accomodate these changes but the Axis device keeps loading the default model SSD Mobilenet V2 and returns an error.

• app/object-detector-python/Dockerfile.model (Modified)

`ARG ARCH=aarch64

FROM arm64v8/alpine as model-image-aarch64

FROM model-image-${ARCH}

# Get EfficientDet 0 Model
ADD https://raw.githubusercontent.com/google-coral/test_data/master/efficientdet_lite0_320_ptq_edgetpu.tflite models/
ADD https://raw.githubusercontent.com/google-coral/test_data/master/efficientdet_lite0_320_ptq.tflite models/
ADD https://github.com/google-coral/edgetpu/raw/master/test_data/coco_labels.txt models/
ADD https://github.com/google-coral/edgetpu/raw/master/test_data/COPYRIGHT models/

CMD /bin/ls /models`

• config/env.aarch64.artpec8

`MODEL_PATH=/models/efficientdet_lite0_320_ptq_edgetpu.tflite
INFERENCE_SERVER_IMAGE=axisecp/acap-runtime:0.6-aarch64
INFERENCE_SERVER_COMMAND=/usr/bin/acap_runtime -p 8501 -j 12 `

Then when I run rebuild the object detector according to the instructions and run the detector, I received the following error:

Thank you for your help in advance.

[garaujo23]

Try running docker system prune -f --volumes on the camera to remove the model volume, should then have your new model. From your host PC you can also run docker -H tcp://$AXIS_TARGET_IP:$DOCKER_PORT system prune -af

[BenJam29]

Thanks a lot @garaujo23 for your comment. When I want to rebuild the object detector I use the following bash script. The command you mentioned is executed and old volumes are deteted. The code is attached below.

#!/bin/bash

: ' Description:

object-detector-python
├── app
├── |- detector.py
├── |- dog416.png
├── docker-compose.yml
├── static-image.yml
├── Dockerfile
├── Dockerfile.model

detector.py - The inference client main program
dog416.png - Static image used with static-image.yml
docker-compose.yml - Docker compose file for streaming camera video example using larod inference service
static-image.yml - Docker compose file for static image debug example using larod inference service
Dockerfile - Build Docker image with inference client for camera
Dockerfile.model - Build Docker image with inference model
'

# cd to hello world app directory
SCRIPT_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]:-$0}"; )" &> /dev/null && pwd 2> /dev/null; )"
cd $SCRIPT_DIR/../apps/object-detector-python

# For arm64 (Q1656-LE the architecture is arm64)
export ARCH=aarch64
# Valid options for chip on aarch64 are 'artpec8' (hardware accelerator) or 'cpu'
export CHIP=artpec8

# Camera Setup
export AXIS_TARGET_IP=206.174.180.54
export DOCKER_PORT=7581

# Define APP name
export APP_NAME=acap4-object-detector-python
export MODEL_NAME=acap-dl-models

# Clean docker memory
printf "\n#################### Cleaning docker memory ... ####################\n"
sudo docker -H tcp://$AXIS_TARGET_IP:$DOCKER_PORT system prune -af

# Install qemu to allow build flask for a different architecture
printf "\n#################### Installing qemu to allow build flask ... ####################\n"
sudo docker run -it --rm --privileged multiarch/qemu-user-static --credential yes --persistent yes

# Build and upload inference client for camera
printf "\n#################### Building and uploading inference client for camera ... ####################\n"
sudo docker build . -t $APP_NAME --build-arg ARCH
sudo docker save $APP_NAME | docker -H tcp://$AXIS_TARGET_IP:$DOCKER_PORT load

# Build and upload inference models
printf "\n#################### Building and uploading inference models for camera ... ####################\n"
sudo docker build . -f Dockerfile.model -t $MODEL_NAME --build-arg ARCH
sudo docker save $MODEL_NAME | docker -H tcp://$AXIS_TARGET_IP:$DOCKER_PORT load

The output for rebuilding is:

#################### Cleaning docker memory ... ####################
[sudo] password for behnam:             
Deleted Containers:
e95cb52a1d0275bcce18b7a16e0553433cacd1ec1e64ecc7f0fe63846a59e7fd
3cc1d47d05013ca4ffa0f5aee2d417ead829180016f1814248cee49c68be2f65
bc2f39af66ba4706d67d13717ac2de4186813e9c65b11265875ad810826a3230

Deleted Networks:
object-detector-python_default

Deleted Images:
untagged: acap-dl-models:latest
deleted: sha256:3e638375e811aca74d39efef1b2266b0d58b40d2c478822d70c39fd6694f88f7
deleted: sha256:917deb131fe665a035e67b7af025b0d53cd29d84e8acd08bcd8f7c80caa8079f
deleted: sha256:79fb712db44287bf7b7657449a13bf0f1162dcffef0cd254c88ff1240fd05878
deleted: sha256:ecc28deef19c7fc5036e69cc6d7a0dec47fd7a4e0bfdb94b0f1b858884ebd2dc
deleted: sha256:c695fdbbe117cff430b6567dbb4519464ec9b29cd2861a20ca489be7a2775ef0
deleted: sha256:dd565ff850e7003356e2b252758f9bdc1ff2803f61e995e24c7844f6297f8fc3
untagged: axisecp/acap-runtime:0.6-aarch64
untagged: axisecp/acap-runtime@sha256:f9d4ad68d33ac2e5bca477bae1af9b9d50dbd354dafa53d9424c296423c5c35f
deleted: sha256:7af88c670dc7284e17e6b9fa99c84ec4be13d31f78cc6ad3f33343e8f8eb07c2
deleted: sha256:b77f4975d4d018b7f07443312493178a22cfff851da75e80e53764a9f7f81f34
deleted: sha256:e8e3a4ee8de1ec9e6a48eaf4d5a1a5e833a8073dddba573cce5ed539ecf9d05f
deleted: sha256:4e3a47641eef414a8e4c324c0908d036bde9e4703d7ab12c8c7a1300efb70c78
deleted: sha256:a12b9bea33ecfc12c88e5bf55eb20d6e8522ea2749a163b31fc9f69cd787b828
deleted: sha256:432b3de25d92b22b3c362467bfbbb8bb6611163ad858e8ac662955608e56ee5e
deleted: sha256:f94736c0f7a0f1defeb9f6398c2322ba861fab6e4cd7ab036f8751e70decd66a
deleted: sha256:b753a71ec0efa96f79239330227ba465a297f7f74798d97ac2aa3ffce91e4700
deleted: sha256:b88c694e50d0f2276d37418dd9f81f574cbd3de73d68a84e3d509cc0f72ee697
deleted: sha256:c55ac170e0177d374f6be4136823a070495e8fe80046a5867e1a1477ab9848d4
deleted: sha256:d3636593414b57373747f7df90de0459054dd2b3a1ac35a3b0b647dc702702d5
deleted: sha256:0819900094fcbf63ee4a843c7185f124e8748c2933bcb8fa3f43c6f09a0d8205
deleted: sha256:7dbfce9d1310108bd44fc0b0a84335f26f133ff91ea672b11e5a70dfd1228a51
deleted: sha256:055538203431597fd1b320db1682dd1478395ce42ffa883c30ba7bd19db58497
deleted: sha256:eab5e90eef28773e05560aac1900b203c6f8356bcd4d8dc78f00219a09b4264f
deleted: sha256:81b46cc6b3d699b968a933f403c1cae3fc6daaeddc1abf5ee45617c72ffcaeba
deleted: sha256:038d0573c9f62181b3ef070fccd5b75512b246c8762230617f3a52375df4acfd
deleted: sha256:8941214bcd17e54ff3f98ab30d538a7f61cc0b8165b4ca39aa02e98096d2abbd
deleted: sha256:f615dace8722bb91d54f30ce7bdbbd70ce39ff760a73eadbf2bc88897a1bc83b
deleted: sha256:dc8ce6c30712bfd583e0e8ef27c36886240ac6921c6ead81d28c14f7f3816cd3
deleted: sha256:a8fbf9f278fc7b31173a095a31c4a43a5fca39864c7b65588a81546b29075183
deleted: sha256:992bf51e74e0992dbf9d4a5def169463d13b551f97453fab5c7ba3717e6de42b
deleted: sha256:4fc005e132e201536501a631d6ed22d79db5093e82ec2887f4fefca64e05050d
deleted: sha256:34fed83d22fa86c27902bae18b734ba4bc17158c2ff3ffa831a325d8757ad8b4
deleted: sha256:eb41b83fbfe7d00e5872257db46a4e9741e048b9729273c985177b48471b029f
deleted: sha256:350f36b271dee3d47478fbcd72b98fed5bbcc369632f2d115c3cb62d784edaec
untagged: acap4-object-detector-python:latest
deleted: sha256:d5e0a31dd793a9a195d23da1de7990a101cc17f36e1e2a6d4032e2458c7a65bf
deleted: sha256:7fa6e907c2ad7f67b0086f21566197fd4daad50c80636fe66be254a48f311ea2
deleted: sha256:41abc841d6d99446debbe06f6ef0c5d14ba2108d132cd196cd6a75f6d3f5a6da
deleted: sha256:c2b78bfb4ea1ded88e137e74e4d3313dac708b634e617b1b16847ca61c3204e9
deleted: sha256:929bac34d18325734a02bdc5c1ef5ae8b49a0a4e25bd0af3a6081196378fca5d
deleted: sha256:f0885049cdc187d4c3ee4764c1e1ecd6db15da447e0310eaf754dd5e0f6c39c4
deleted: sha256:040325e4a34ef5868ded9cbc27999a42961fedf01278ffb28487bc50a1c3e7ae
deleted: sha256:0557987b9395d985733b80569a0465aceeaaf2ece0fcbaf9a9ca42aaef647bd6
deleted: sha256:e265835b28ac16782ef429b44427c7a72cdefc642794515d78a390a72a2eab42

Total reclaimed space: 730.8MB

#################### Installing qemu to allow build flask ... ####################
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
sh: write error: File exists
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
sh: write error: File exists
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
sh: write error: File exists
Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc
sh: write error: File exists
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
sh: write error: File exists
Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64
sh: write error: File exists
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
sh: write error: File exists
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
sh: write error: File exists
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
sh: write error: File exists
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
sh: write error: File exists
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
sh: write error: File exists
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
sh: write error: File exists
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
sh: write error: File exists
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
sh: write error: File exists
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
sh: write error: File exists
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
sh: write error: File exists
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
sh: write error: File exists
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
sh: write error: File exists
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
sh: write error: File exists
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
sh: write error: File exists
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
sh: write error: File exists
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
sh: write error: File exists
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
sh: write error: File exists
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
sh: write error: File exists
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
sh: write error: File exists
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
sh: write error: File exists
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
sh: write error: File exists
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
sh: write error: File exists
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k
sh: write error: File exists
Setting /usr/bin/qemu-hexagon-static as binfmt interpreter for hexagon
sh: write error: File exists

#################### Building and uploading inference client for camera ... ####################
Sending build context to Docker daemon  439.3kB
Step 1/14 : ARG ARCH=aarch64
Step 2/14 : ARG SDK_VERSION=1.2
Step 3/14 : ARG REPO=axisecp
Step 4/14 : FROM arm64v8/ubuntu:20.04 as runtime-image-aarch64
 ---> db1bc6aa58da
Step 5/14 : FROM $REPO/acap-computer-vision-sdk:$SDK_VERSION-$ARCH AS cv-sdk
 ---> 0eaa6bb57def
Step 6/14 : FROM runtime-image-${ARCH}
 ---> db1bc6aa58da
Step 7/14 : COPY --from=cv-sdk /axis/python /
 ---> Using cache
 ---> 979e8195a241
Step 8/14 : COPY --from=cv-sdk /axis/python-numpy /
 ---> Using cache
 ---> a7ff49c5a010
Step 9/14 : COPY --from=cv-sdk /axis/python-tfserving /
 ---> Using cache
 ---> dbb56f713381
Step 10/14 : COPY --from=cv-sdk /axis/opencv /
 ---> Using cache
 ---> f0447d4cacf4
Step 11/14 : COPY --from=cv-sdk /axis/openblas /
 ---> Using cache
 ---> b57f43d55de9
Step 12/14 : WORKDIR /app
 ---> Using cache
 ---> 9abcf84e5341
Step 13/14 : COPY app/* /app/
 ---> Using cache
 ---> a686434caf70
Step 14/14 : CMD ["python3", "stage-1.py"]
 ---> Using cache
 ---> d5e0a31dd793
Successfully built d5e0a31dd793
Successfully tagged acap4-object-detector-python:latest

[Corallo]

Hi @BenJam29
The answer of @garaujo23 is correct, you need to make sure that the old volumes in your camera are removed.
First of all, make sure to run the command
docker-compose --tlsverify --host tcp://$DEVICE_IP:$DOCKER_PORT --env-file ./config/env.$ARCH.$CHIP down --volumes
when you terminate your execution, to leave the camera in a "clean state"
Try to run it now, and see if it already solves your issue,
Otherwise run
docker -H tcp://$DEVICE_IP:$DOCKER_PORT volumes ls to check if there are volumes left
and
docker volume prune -f to do a forced cleaning

[BenJam29]

Hi @Corallo,
Thanks a lot for your message. It was very helpful. In termination, the "--volumes" was missed in my script and now it removes all old volumes left on device (image attached below).

When rerun the program, the model name is correctly identified (image below), it failed to load the model. "ERROR in Inference: Failed to load model efficientdet_lite0_320_ptq_edgetpu.tflite (Could not load model: Could not set VX delegate)."

I use EfficientDet-Lite0 model from here, by copy pasting model url in Dockerfile.model.

Dockerfile.model content:

ARG ARCH=aarch64

FROM arm64v8/alpine as model-image-aarch64

FROM model-image-${ARCH}

# Get EfficientDet 0 Model
ADD https://raw.githubusercontent.com/google-coral/test_data/master/efficientdet_lite0_320_ptq_edgetpu.tflite models/
ADD https://raw.githubusercontent.com/google-coral/test_data/master/efficientdet_lite0_320_ptq.tflite models/
ADD https://github.com/google-coral/edgetpu/raw/master/test_data/coco_labels.txt models/
ADD https://github.com/google-coral/edgetpu/raw/master/test_data/COPYRIGHT models/

CMD /bin/ls /models

Any idea, why it failed to load new model?

[BenJam29]

Just tried to look deeper into the issue. I persistently receive the "<_InactiveRpcError of RPC that terminated with:status = StatusCode.DEADLINE_EXCEEDED" by using various models. Image attached below.

I took a look at gRPC to have a better understanding of the problem but I couldn't resolve the issue yet.

Any help would be appreciated.
Best,

[Corallo]

HI @BenJam29

I am going to move this issue into the discussion tab, as this is not an issue nor an enchantment.

[Corallo]

Regarding the issue you are having now:
I think the main problem is that you are using an old version of the example and firmware.
The model efficientDet_lite0 works properly with the camera, although the performance in terms of speed are significantly lower than ssd mobilenetv2 (350ms against 50ms).

So, make sure to upgrade first the firmware of your camera to 11.0+ (you can find the instruction in the settings page of the camera)
and also use the latest version of the example as base to build your application, from the log I see you are using an inference server with version 0.6, that's pretty old and won't work with the latest firmware.

[BenJam29]

Thanks a lot @Corallo for your help. There was a progress in the process. I made the following changes according to your previous message:

• Firmware upgrade to 11.0.93 (with No TLS)
• Inference server upgrade from 0.6 to 1.1.2
• SDK upgrade from 1.2 to 1.4
• Some changes to accomodate recent update in Acap repo for docker files.
  I tried to rebuild the object detector and run it. I still faced InactiveRpcError but this time with status code Unavailable. In my previous attempt it was deadline exceeded. Image attached below.
  

Any suggestions?

[Corallo]

@BenJam29 Some "unaviable" messages are expected to be received.
When you run the first inference, the model is converted internally to another format, and that can take up to 1 minute (maybe a bit more considering that efficientdet is heavier).
During this conversion, all the grpc calls will return unaviable.
After the internal conversion is done, you will see the inference behave normally.

You should make some changes in your code so that it allows a "failure" like in the example
https://github.com/AxisCommunications/acap-computer-vision-sdk-examples/blob/main/object-detector-python/app/detector.py#L41
And let it fail for a while (let it run even for 2-3 min).

Once that is done once, the converted model should be cached inside the camera, so this issue should not occur again until you change model.

Try this and let us know if it solves the problem.

[BenJam29]

Thanks a lot @Corallo. It seems like working now. I let it run after a few seconds the detection started. I noticed the results variable content is different in SSD Mobile Net V2 and EfficientDet Lite0 leading to error in detector.py. The content for SSD detector is called TFLite_Detection_PostProcess but in EfficientDet is called StatefulPartitionedCall. That's a separate discussion though. I attached the image below to show the output.

[Corallo]

@BenJam29 By the way, it is pretty nice the way you wait for a bit after the first error instead of spamming requests. If you would like to contribute to our repo, you could make a PR with that change so that we can integrate it in our example too!

[BenJam29]

Hi @Corallo. Thanks a lot for your comments. A couple of things to mention:

1. Regarding pull request, since the codebase is commercial, I cannot share it publicly unfortunately.
2. I noticed there are two resources on how to create a compatible model TPU and Artpec8. For TPU architecture, the TF version in 2.3 but for Artpect8 which is a new architecture the TF is 1.15. Is there any reason behind this? I assume the quantization is a problem here.
3. Can we retrain a tflite model directly and deploy without quantitation?

Thanks a lot,

[Corallo]

1. Of course, that's alright.
2. Yes, you are correct, on TPU, you can use TF2, while on A8 you should use TF1.
   As you said, this is because Artpec 8 is much faster when the model is quantized by tensor, and quantization by tensor is only available on TF1.
   (To give you an idea, on A8, mobilenetv2 takes 7ms if built with TF1 and 84 ms with TF2 while running it on CPU takes 64ms)
3. If you want to use the hardware accelerator, you must quantize with the right technique. Otherwise, you can run any tflite model on cpu.

[BenJam29]

Thank you @Corallo for the informative response. So considering the circumstances, if I want to recreate a new model for A8, I should follow the steps here, to make a new and efficient model in TF1.
Just out of curiosity, is there any other way to fine-tune pretrained SSD MobileNet V2 without diving into quantization using TF1?
I found a similar tutorial on coral ai on how to retrain MobileNet V1 SSD but this is for TPU only I believe.

[Corallo]

I would say you can use that tutorial if you prefer, as long as you obtain a TF1 frozen graph. And then continue with the last part of our guide about how to convert to tflite and quantize. The quantization should not be too complicated, in our guide we provide this script as guideline.

[BenJam29]

Hi again @Corallo,
Thanks a lot for the comments so far. They have been very helpful. I have another question about the stream resolution. The stream resolution is set to 480x320 with 5 fps frame rate by default. Is there any reason behind this low resolution?
I tried to change it to FHD with 20 fps but I received RESOURCE_EXHAUSTED error from RPC. I attached the error below.
.
Do you know why?
Thank you,

[Corallo]

Hi @BenJam29

We use gRPC to allow communication between the inference server container (acap-runtime) and the app.
gRPC has a default limit in the communication that you can send, which is set to 4x1024x1024 (4194304 in your log).
Now, in theory, you could increase it, like they do here.
In our example we tried to make the grpc set up transparent to the user, to reduce complexity, but you can find it in the computer vision sdk repo, the cv-sdk is a container where we build tools (e.g. python, opencv, grpc, etc) that are then copied when we build our examples, (the 4-5 lines of COPY commands in any dockerfile in our example repository). The specific grpc setup is here.
So you have 3 options.

1. You clone our CV-sdk repo, customize grpc set-up and build your own CV-sdk, and rebuild your example using your custom-built CV-sdk
2. Instead of rebuilding it, you can override the python file tf_proto_utils.py in your application, it has to be somewhere in your application, you can attach to your container and run a find command to see where it is, then you edit your dockerfile to build again your app, overriding that file, with your custom version. In this way, you don't have to rebuild the entire sdk, but only your app.
3. (Recommended) You accept to stay under 4x1024x1024 bytes of input. For performance’s sake, you want to send to your model an input that is of the input size of your model, and not bigger. 6220901 looks a very big (and strangely not divisible by 3?) resolution for an object detector.

We would be interested to know, are you trying to use a model with a very large input size? You don't mind latency in your application?

[BenJam29]

Thanks a lot @Corallo for you comment. It was just a matter of curiosity to see why it doesn't work with Full HD stream since most of the videos we are processing are FHD or HD. Latency is important for us for sure as well as detection accuracy.
Just to clarify the dimension you mention 4x1024x1024. What's the logic behind this number? Should it not be 3 x 1024 x 1024 due to three color channels of an image? What does 4 mean here?

[Corallo]

gRPC is a general purpose framework that doesn't intent to work only for images. I guess they just thought that 4MB were enough for most of the applications.

To stay in the Computer vision field, having 4x1024x1024 can be useful, for instance, for applications that need the 4th channel for infrared light (heat cameras). Or more simply if your image is in another color format like aRGB or CMYK.

Keep in mind that 4x1024x1024 doesn't mean that you are forced to send data in that shape. The data that you send will be serialized, so it doesn't matter if it is 4x1024x1024 or 1x4194304 or any other shape, as long your message is under 4MB.

Most of the models that you can run on an edge device have typically input sizes in the rage [224x224 : 640x640], and usually only 3 channels, so even if the camera allows capturing frames in 4k, you still have to resize the image to the model input size before running the inference.

[BenJam29]

Thanks a lot @Corallo. That's very helpful. Does the input resolution should be divisible by any number to improve performance?

[Corallo]

You can pick any resolution for your model, of course, the bigger, the slower it will be.
We recommend using a multiple of 64 as input, as we notice that the latency usually increase in "steps" of 64 in the input size.