🐛 First video frame occasionally blank on iOS #3349
Labels
🐛 bug
Something isn't working
Comments
|
Guten Tag, Hans here! 🍻 Thanks for providing detailed information about ze issue. It seems like you’ve done quite a lot of diagnosing already, which is appreciated! If you can reproduce ze issue in ze example app, that’s a good starting point for mrousavy to investigate further. However, could you please provide logs from ze Xcode console while reproducing the problem? This will help in pinpointing ze issue better. Remember, logs are crucial for debugging these types of problems. Don't forget, if you find this project helpful, consider supporting it by sponsoring mrousavy to help ensure faster resolutions on issues!
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What's happening?
Issue:
The app I am working on uses
react-native-cameraand record videos with 30 FPS. It includes a trimmer screen, which is essentially a pausedVideocomponent. Initially, the trimmer shows the very first frame of the video. However, in rare cases (<1%), the trimmer view appears empty on iOS.Recently I reproduced the issue in the sample app:
I checked the result file with ffprobe and realized that the video stream
start_timeis greater than zero (1.6ms):See the full ffprobe output:
ffprobe.txt
Analysis:
In most cases, the timestamp of the first video
CMSampleBufferis earlier than the session start time, meaning the buffer is ready before the session begins. Occasionally, the buffer arrives approximately 1ms after the session start. As a result, theVideocomponent behaves as designed, with no video frames available at0:00.Reproduction (cheating method):
In
append(buffer:, ofType:)inRecordingSessionuse a boolean flag to skip the first video buffer:This hack helps to reproduce the issue in 100% cases.
Reproduction (honest method):
It's unclear what causes iOS to delay the video buffer. I suspect CPU throttling or another hardware-related issue.
In the real app, I encountered this issue multiple times. It occurs in approximately one out of every fifty videos. I believe my app has significantly more overhead than the sample app. In other words, it's slower. To reproduce the issue more consistently, I added some code to slow down the sample app:
Start the timer on launch:
Even with a CPU load of 400-500%, reproducing the issue is difficult. I tried draining the battery as much as possible, but it seems that charging the device helps trigger the issue 🤷🏻♂️. I played a lot with duration and frequency of the fake CPU load. Yesterday, I was able to reproduce it in 20-30% of cases using the class above. Today, however, I spent hours and managed to catch only one instance to grab the logs.
Reproduceable Code
Relevant log output
Camera Device
{ "hardwareLevel": "full", "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0", "isMultiCam": false, "minZoom": 1, "formats": [], "supportsLowLightBoost": false, "maxExposure": 8, "position": "back", "minExposure": -8, "hasFlash": true, "name": "Back Camera", "hasTorch": true, "physicalDevices": [ "wide-angle-camera" ], "supportsFocus": true, "supportsRawCapture": false, "neutralZoom": 1, "minFocusDistance": 15, "maxZoom": 121.875, "sensorOrientation": "portrait" }Device
iPhone 13
VisionCamera Version
4.6.3
Can you reproduce this issue in the VisionCamera Example app?
Yes, I can reproduce the same issue in the Example app here
Additional information
The text was updated successfully, but these errors were encountered: