You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The app I am working on uses react-native-camera and record videos with 30 FPS. It includes a trimmer screen, which is essentially a paused Video component. Initially, the trimmer shows the very first frame of the video. However, in rare cases (<1%), the trimmer view appears empty on iOS.
Recently I reproduced the issue in the sample app:
I checked the result file with ffprobe and realized that the video stream start_time is greater than zero (1.6ms):
In most cases, the timestamp of the first video CMSampleBuffer is earlier than the session start time, meaning the buffer is ready before the session begins. Occasionally, the buffer arrives approximately 1ms after the session start. As a result, the Video component behaves as designed, with no video frames available at 0:00.
Reproduction (cheating method):
In append(buffer:, ofType:) in RecordingSession use a boolean flag to skip the first video buffer:
if type ==.video, !hasFirstBuffer {
hasFirstBuffer = true
return}
This hack helps to reproduce the issue in 100% cases.
Reproduction (honest method):
It's unclear what causes iOS to delay the video buffer. I suspect CPU throttling or another hardware-related issue.
In the real app, I encountered this issue multiple times. It occurs in approximately one out of every fifty videos. I believe my app has significantly more overhead than the sample app. In other words, it's slower. To reproduce the issue more consistently, I added some code to slow down the sample app:
privateclassThrottler{privatevartimer:Timer?func start(interval:TimeInterval, duration:TimeInterval){stop()
timer =Timer.scheduledTimer(withTimeInterval: interval, repeats: true){ _ in
for _ in 0..<10{DispatchQueue.global().async{self.calc(duration)}}}}func stop(){
timer?.invalidate()
timer =nil}privatefunc calc(_ duration:TimeInterval){letendTime=Date().addingTimeInterval(duration)
while Date()< endTime {
_ =sqrt(Double.random(in:0..<1))}}}
Start the timer on launch:
private var throttler = Throttler()
...
throttler.start(interval: 0.02, duration: 0.012)
Even with a CPU load of 400-500%, reproducing the issue is difficult. I tried draining the battery as much as possible, but it seems that charging the device helps trigger the issue 🤷🏻♂️. I played a lot with duration and frequency of the fake CPU load. Yesterday, I was able to reproduce it in 20-30% of cases using the class above. Today, however, I spent hours and managed to catch only one instance to grab the logs.
Reproduceable Code
// The sample code as is but no logs and frame processor:<ReanimatedCamerastyle={StyleSheet.absoluteFill}device={device}isActive={isActive}ref={camera}onInitialized={onInitialized}onError={onError}format={format}fps={fps}photoHdr={photoHdr}videoHdr={videoHdr}photoQualityBalance="quality"lowLightBoost={device.supportsLowLightBoost&&enableNightMode}enableZoomGesture={false}animatedProps={cameraAnimatedProps}exposure={0}enableFpsGraph={true}outputOrientation="device"photo={true}video={true}audio={microphone.hasPermission}enableLocation={location.hasPermission}/>// Also the video is always paused on `MediaPage`<Videosource={source}style={StyleSheet.absoluteFill}paused={true}.../>
Guten Tag, Hans here! 🍻 Thanks for providing detailed information about ze issue. It seems like you’ve done quite a lot of diagnosing already, which is appreciated!
If you can reproduce ze issue in ze example app, that’s a good starting point for mrousavy to investigate further. However, could you please provide logs from ze Xcode console while reproducing the problem? This will help in pinpointing ze issue better. Remember, logs are crucial for debugging these types of problems.
Don't forget, if you find this project helpful, consider supporting it by sponsoring mrousavy to help ensure faster resolutions on issues!
Note: If you think I made a mistake, please ping @mrousavy to take a look.
What's happening?
Issue:
The app I am working on uses
react-native-camera
and record videos with 30 FPS. It includes a trimmer screen, which is essentially a pausedVideo
component. Initially, the trimmer shows the very first frame of the video. However, in rare cases (<1%), the trimmer view appears empty on iOS.Recently I reproduced the issue in the sample app:
I checked the result file with ffprobe and realized that the video stream
start_time
is greater than zero (1.6ms):See the full ffprobe output:
ffprobe.txt
Analysis:
In most cases, the timestamp of the first video
CMSampleBuffer
is earlier than the session start time, meaning the buffer is ready before the session begins. Occasionally, the buffer arrives approximately 1ms after the session start. As a result, theVideo
component behaves as designed, with no video frames available at0:00
.Reproduction (cheating method):
In
append(buffer:, ofType:)
inRecordingSession
use a boolean flag to skip the first video buffer:This hack helps to reproduce the issue in 100% cases.
Reproduction (honest method):
It's unclear what causes iOS to delay the video buffer. I suspect CPU throttling or another hardware-related issue.
In the real app, I encountered this issue multiple times. It occurs in approximately one out of every fifty videos. I believe my app has significantly more overhead than the sample app. In other words, it's slower. To reproduce the issue more consistently, I added some code to slow down the sample app:
Start the timer on launch:
Even with a CPU load of 400-500%, reproducing the issue is difficult. I tried draining the battery as much as possible, but it seems that charging the device helps trigger the issue 🤷🏻♂️. I played a lot with duration and frequency of the fake CPU load. Yesterday, I was able to reproduce it in 20-30% of cases using the class above. Today, however, I spent hours and managed to catch only one instance to grab the logs.
Reproduceable Code
Relevant log output
Camera Device
Device
iPhone 13
VisionCamera Version
4.6.3
Can you reproduce this issue in the VisionCamera Example app?
Yes, I can reproduce the same issue in the Example app here
Additional information
The text was updated successfully, but these errors were encountered: