Using MediaRecorder in Ionic

[GertBoer]

I am building an app in Ionic which makes use of the IOSRTC plugin for using the WebRTC APIs.

I try to record and store videos. With the MediaDevices.getUserMedia() method I successfully can stream video to a video element.

The next step is to save and store the video stream with the MediaRecorder method.

this.mediaRecorder = new MediaRecorder(this.stream, options);

It seems that this method is available and can be used in my Ionic app (iPhone / IOS15). However when the MediaRecorder is recording and the ondataavailable is triggered, the media data is always 0.

this.mediaRecorder.ondataavailable = (event) => {
    console.log(event.data.size) // <-- this is always 0
    if (event.data && event.data.size > 0) {
       chunks.push(event.data);
    }
};

In the Safari browser on my laptop the event.data.size is not 0. But on my builded Ionic app it is. Do any of you know what the problem could be?

(I don't know if this question belongs here)

[hthetiot]

TLDR: use Cordova's Media Capture Plugin.

Unfortunalty iosRTC WebRTC implementation does not support MediaRecorder and passing a MediaStream from iosRTC to MediaRecorder will simply not works.
iOS WebRTC implementation is custom (It's a giant Polyfill basicly) and would require to implement a SHIM for MediaRecorder to works with iosRTC MediaStream.

Alternativly if you want video only, you can implement video capture using "videoElement.render.save" like that:

var canvasEl;
function TestMediaRenderCatpure(videoEl) {
  canvasEl = document.createElement('canvas');
  var ctx = canvasEl.getContext("2d");
  canvasEl.width = "100";
  canvasEl.height = "100";
  canvasEl.style.position = 'absolute';
  canvasEl.style.left = 0;
  canvasEl.style.bottom = 0;
  appContainer.appendChild(canvasEl);
  var image = new Image();
  image.onload = function() {
    ctx.drawImage(image, 0, 0);
  };
  videoEl.render.save(function (data) {
    image.src = "data:image/jpg;base64," + data;
  });
} 

Source: https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/extra/renderer-and-libwebrtc-tests.js#L424

Then use captureStream on Canvas to get "Native" MediaStream and pass it to MediaRecorder video recording.
Alternativly you can also implement your webview content over HTTPS to bypass this plugin and use Native WebRTC.

[GertBoer]

I have a iPhone 13 Pro. I have tried both requestAnimationFrame and setTimeout. Both are stuttering. I don't think that video capture through the webAPI's are the solution for me. I have now implemented the Cordova's Media Capture Plugin. Downside is that I don't have much control over the working of this plugin.

A little bit of topic, but do you have maybe an other suggestions for a video capturing solution?

[hthetiot]

A little bit of topic, but do you have maybe an other suggestions for a video capturing solution?

Not for now, may be you can try to not use this plugin and use Native WebRTC by serving your app content through https since GetUserMedia in WkWebView available starting iOS 14.3+

[hthetiot]

Alternatively using this plugin and reducing the video quality using GetUserMedia video contraints can help with performance issues.

[hthetiot]

use Cordova's Media Capture Plugin is the solution.

[webbpage]

Not for now, may be you can try to not use this plugin and use Native WebRTC by serving your app content through https since GetUserMedia in WkWebView available starting iOS 14.3+

The alternative of using https vs this plugin is very attractive especially for those interested in MacOS support (not yet supported 697) The problem is that using https in Cordova/Capacitor iOS apps is not as easy as it seems.

https://forum.ionicframework.com/t/https-for-the-app-server-protocol-instead-of-capacitor-on-ios/200116

https://forum.ionicframework.com/t/is-that-possible-to-change-ios-capacitor-scheme-from-capacitor-to-http-or-https/218036

https://forum.ionicframework.com/t/capacitor-hostname-config-does-not-work-for-ios/218454

If others have found a way to accomplish https on iOS, please share!