Multistream Quickstart Guide

[Work in progress...]

With Multistream, developers gain greater flexibility when displaying remote media streams. Instead of receiving a single stream featuring all participants, Multistream enables the showcasing of multiple streams simultaneously. This means you can easily view each remote participant's stream individually, enhancing clarity and engagement during collaborative sessions.

This guide provides a quick glimpse into consuming multistream in Webex Meetings using the Webex Javascript SDK. You'll find code snippets below that can quickly get you set up to use this feature.

To gain a deeper understanding of the upcoming concepts, please consult the Concepts and Terminology section in the comprehensive guide.

To work with the multistream feature please use the version 3.0.0, use the below commands or script:

npm install webex@3.0.0

(or)

yarn add webex@3.0.0

(or)

<script defer src="https://unpkg.com/webex@3.0.0/umd/webex.min.js"></script>

To integrate Multistream functionality into your meetings, ensure you include the enableMultistream flag within the joinOptions parameter when utilizing either the meeting.join() or meeting.joinWithMedia() methods. This straightforward addition seamlessly activates Multistream capabilities, allowing for efficient incorporation into your meetings.

e.g.

const joinOptions = {
  enableMultistream: true,
  ...
};

await meeting.join(joinOptions);

// or

await meeting.joinWithMedia({joinOptions, mediaOptions});

Note: For additional details regarding joining a meeting, please consult this documentation.

Setup Remote Media Event Listeners

Once you join a meeting successfully, the meetings backend will start sending the remote streams which can be received by listening to the following two events for audio and video

1. media:remoteAudio:created - For remote audio streams
2. media:remoteVideo:layoutChanged - For remote video streams like active speaker, participants and screen share

Attach Audio Streams

Sample HTML code for audio elements

<!-- Audio elements -->
<audio id="multistream-remote-audio-0" class="multistream-remote-audio" autoplay></audio>
<audio id="multistream-remote-audio-1" class="multistream-remote-audio" autoplay></audio>
<audio id="multistream-remote-audio-2" class="multistream-remote-audio" autoplay></audio>

Note: You can only receive a maximum of 3 audio streams from active speakers at a time, so there's no need to create additional elements.

Sample JavaScript code for binding remote audio streams to HTML elements

meeting.on('media:remoteAudio:created', (audioMediaGroup) => {
  audioMediaGroup.getRemoteMedia().forEach((media, index) => {
    document.getElementsByClassName('multistream-remote-audio')[index].srcObject = media.stream;
  });
});

Attach Video Streams

const activeSpeakerVideoElems = [];
const memberVideoElems = [];

meeting.on('media:remoteVideo:layoutChanged', ({
  layoutId, activeSpeakerVideoPanes, memberVideoPanes, screenShareVideo
}) => {
  for (const [groupId, group] of Object.entries(activeSpeakerVideoPanes)) {
    group.getRemoteMedia().forEach((remoteMedia, index) => {
      // Attach the "remoteMedia.stream" of active speakers to video elements
      if(remoteMedia.sourceState === 'live') {
        // Follow through the Layout creation section for the definition of createVideoElement()
        activeSpeakerVideoElems.push(createVideoElement(remoteMedia.stream));
      }
    });
  }

  // staged layout is the only one we use that has memberVideoPanes defined. Read through the comprehensive guide for more details.
  for (const [paneId, remoteMedia] of Object.entries(memberVideoPanes)) {
    // Attach the "remoteMedia.stream" of member videos to video elements
    if(remoteMedia.sourceState === 'live') {
      // Follow through the Layout creation section for the definition of createVideoElement()
      memberVideoElems.push(createVideoElement(remoteMedia.stream));
    }
  }

  // Follow through the Layout creation section for the definition
  updateTheLayout();

  if (screenShareVideo) {
    // Attach the "screenShareVideo.stream" to a video element
    remoteScreenshareElm.srcObject = screenShareVideo.stream;
  }
});

Build the Layout

The layout is essentially about how the various streams are positioned and displayed on the user interface, ensuring they are organized in a way that is both visually appealing and easy to navigate.

In order to create the layout with remote streams, follow through the steps below

• Create a container element (say "div") to contain all the remote video elements
• For each streams
  • Create an HTML "video" element and attach the remoteMedia.stream
  • Add elements for name label, overlay and overlay text elements to display participant information on top of the video streams - [Optional]
• Append all the video elements to the container element

Sample HTML code showcasing the remote streams.

<!-- Create a container element (say "div") -->
<div id="multistream-remote-video" style="display: flex">
  <!-- All the remote videos will render here -->
</div>

Sample JavaScript code demonstrating the creation of a video element and the dynamic adjustment of the layout.

const remoteVideoContainerElm = document.getElementById('multistream-remote-video');

// Create an html "video" element and attach the `remoteMedia.stream`
function createVideoElement(stream) {
  const videoElement = document.createElement('video');
  videoElement.srcObject = stream;
  videoElement.height = 480;
  videoElement.width = 620;
  videoElement.autoplay = true;
  videoElement.muted = true;

  return videoElement;
}

function updateTheLayout() {
  [...activeSpeakerVideoElems, ...memberVideoElems].forEach((videoElement) => {
    // Append all the video elements to the container element
    remoteVideoContainerElm.appendChild(videoElement);
  });
}

Now that you've witnessed multistream in action, for a more thorough understanding and access to all available features, please delve into the Multistream Comprehensive Guide.

Kitchen Sink App

To play around with more features of multistream, please check our Meeting Samples App.

In the Manage Meeting section, select the checkbox labeled Use a multistream connection (which automatically sets enableMultistream to true and passes it to the join functions) before clicking on Join Meeting or Join with Media. Note that enabling this checkbox will replace the Remote Video fieldset with Multistream Remote Videos under Streams section.