Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Intrinsic camera calibration] Extremely high delay of up to 4s on the camera image screen of the calibration software #122

Open
3 tasks done
zymouse opened this issue Sep 1, 2023 · 7 comments

Comments

@zymouse
Copy link

zymouse commented Sep 1, 2023

Please make sure of the following things

  • I have read the documentation.
  • I'm sure there are no duplicate issues or discussions.
  • I'm sure this issue is not fixed in the latest version.

Version

decf391

Describe the bug

image is a great tool, and in combination with apriltag corner detection it also verifies that the internal calibration is great.

During use, it was found that the latency of the camera image was found to be very high.

ultimate

Thank you very much, these calibration tools are great and do a much better job of calibrating the sensors!

@knzo25
Copy link
Collaborator

knzo25 commented Sep 1, 2023

Hi, thanks for using our tools.

The delay is probably due to the detector being too slow for your image size.
What are the dimensions of your image?

Each detector additionally has parameters, which if tuned, will make the latency lower.
The ROS tool among others will resize your image, detect in the smaller image, re-scale, and then refine the detections. I don't really trust that, so I did not implement that on purpose.

@aohsato
The latency when we calibrate C2 cameras is also almost unbearable. Do you have a preference on how to proceed? My current ideas are as follows:

  • Implement the resize strategy as optional, and perform some experiments to check if this approach actually incurs into a bigger reprojection error. Provide config files for C1 and C2 cameras separately
  • Tune the detector parameters to try to reduce the latency for big images. This will probably reduce the detection range, but we won't calibrate with boards over 30m (we don't have the space)
  • Perform detections with queues and multi threading. This will not reduce the latency, but will increase the number of images actually processed and the UI will look more fluid

@aohsato
Copy link
Collaborator

aohsato commented Sep 1, 2023

@knzo25 Thanks, I think that's a good point!

Your three ideas are great, but it might be good to use two different image sizes for some purposes.

  • Resized small image: camera image visualization and corner detection for sampling keyframes.
  • Original size image: corner detection and calibration execution with sampled keyframes.

(keyframe: collected frames actually used for calibration)

What do you think? Is the code change too big?

@knzo25
Copy link
Collaborator

knzo25 commented Sep 1, 2023

Ok, I will do that.
The changes are not to big, but also not that small.
Let's discuss the priority internally.

@zymouse
Please for a while until we implement the changes 🙏

@zymouse
Copy link
Author

zymouse commented Jan 9, 2024

@knzo25 k
Lower the frequency of image_raw, this improved when I lowered it to 10hz

@knzo25
Copy link
Collaborator

knzo25 commented Jan 9, 2024

@zymouse
Hi, thanks for the comment.
We also replicated the issue last week.
Usually we run everything at 10hz, and it runs smoothly for out 1920x1080 camera (it runs slow for higher res cameras), but when I ran it free run (~30Hz) it got laggy.

Since we have the hardware to replicate and check the fix, we should be able to address it soon-ish (by the end of the month, together with the next release)

@ParashRahman
Copy link

I'm experiencing slowness. What is the size parameter units?

@knzo25
Copy link
Collaborator

knzo25 commented Apr 14, 2024

Could you try this branch?
#152

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants