Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix black photo issue on intel MIPI cameras (Bugfix) #1643

Open
wants to merge 26 commits into
base: main
Choose a base branch
from

Conversation

tomli380576
Copy link
Contributor

@tomli380576 tomli380576 commented Dec 8, 2024

Description

The gstreamer pipeline in the current camera_test.py was having trouble with cameras that require a "boot up time" because it was taking the 1st buffer, which is blank for these cameras, that reaches filesink. This PR adds a valve element that dynamically opens while the pipeline is running to allow the source to stay open for a few seconds before the photo is taken.

There was also an issue where <CameraTest>._width and <CameraTest>._height were never updated from 640x480 after we query the supported formats. This breaks on intel MIPI cameras because it only supports 1280x720 and it will complain about not-negotiated caps $\Rightarrow$ Error: Internal data stream error when <Pipeline>.set_state(Gst.State.PLAYING) is called. This is fixed as well but I'm not too sure if this was intentional. Do later tests depend on the image being 640x480?

Resolved issues

#1537

Documentation

The number of seconds to delay can be set by --wait-seconds <num_seconds>. Default is 3.

Tests

  • Original unit tests.
  • Validated on a few machines running 24.04 with intel MIPI cameras and the test runs normally. Machines with USB cameras are nice as usual and pass the test too.

@tomli380576
Copy link
Contributor Author

Validated that valve exists on the default gstreamer installation on 16 and 18 so we should be good

@tomli380576 tomli380576 changed the title Fix black photo issue on intel IPU cameras (Bugfix) Fix black photo issue on intel MIPI cameras (Bugfix) Dec 10, 2024
Copy link

codecov bot commented Dec 10, 2024

Codecov Report

Attention: Patch coverage is 90.69767% with 4 lines in your changes missing coverage. Please review.

Project coverage is 48.88%. Comparing base (1938d60) to head (63fd751).
Report is 12 commits behind head on main.

Files with missing lines Patch % Lines
providers/base/bin/camera_test.py 90.69% 1 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1643      +/-   ##
==========================================
+ Coverage   48.85%   48.88%   +0.03%     
==========================================
  Files         370      370              
  Lines       40231    40261      +30     
  Branches     6793     6798       +5     
==========================================
+ Hits        19654    19681      +27     
- Misses      19857    19858       +1     
- Partials      720      722       +2     
Flag Coverage Δ
provider-base 24.92% <90.69%> (+0.12%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tomli380576 tomli380576 marked this pull request as ready for review December 10, 2024 02:59
@tomli380576 tomli380576 marked this pull request as draft December 10, 2024 03:07
@tomli380576 tomli380576 marked this pull request as ready for review December 17, 2024 07:08
@tomli380576 tomli380576 marked this pull request as draft December 17, 2024 07:11
@tomli380576 tomli380576 marked this pull request as ready for review December 17, 2024 07:59
@tomli380576
Copy link
Contributor Author

tomli380576 commented Dec 19, 2024

Just a quick note on multifilesink: I used it because with the way we are terminating the pipeline (time/num-buffers based EOS), filesink will write extra buffers after the initial valid buffer. Basically it made a file like this:

┌────┬────┬────┬────┐
│ F1 │ F2 │ F3 │ F4 │
└────┴────┴────┴────┘ 

where frame F1 is the actual photo we want, F2-F4 are the extra stuff that filesink kept writing until EOS was sent. This file is still open-able probably because the metadata only tells the image viewer to read up to F1 and just ignore the rest. This can make the image file excessively big if the image dimensions are big or the frame rate is high. I found that 1280x720 photos taken at 30fps was taking up to 7Mb of space when it only needs like ~200Kb.

It is doing extra file writes than necessary though, albeit I haven't found any negative side effects.

There are ways to do exactly 1 file write, for example valve ! jpegenc snapshot=True, valve ! identity eos-after=1, valve ! imagefreeze num-buffers=1, but they don't exist on ubuntu 16's gstreamer :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant