WebRTC: Update and Workarounds

As you’ve probably noticed, we’ve been making lots of progress on our WebRTC implementation, and we expect additional improvements over the next few releases.

We have work in the pipeline to improve audio quality issues (yes, we know we still have some!) and to assist with troubleshooting NAT traversal issues (you can follow the progress in Bug 904622).

Existing limitations

But beyond these upcoming improvements, I’d like to take a moment to look at a couple of our existing limitations that you might have noticed, and offer some advice for writing apps that work within these limitations.

The first issue, described in Bug 857115, is that mozRTCPeerConnection does not currently support renegotiation of an ongoing session. Once a session is set up, its parameters are fixed. In all practicality, this means that you can’t, for example, start an audio-only call and then add video to that same PeerConnection later in that session. We have another similar limitation in that we don’t currently support more than one each audio and video stream on a single PeerConnection (see Bug 784517 and Bug 907339).

Solutions for now

We’re going to fix these limitations as soon as we can, but it’s going to take a few months for our code changes to ride the Firefox train out into release. Until that happens, I want to give you a couple of workarounds so you can continue to use Firefox to make awesome things.

Muting audio and video streams

Media renegotiation has two main use cases: muting and unmuting media in the middle of a session; and adding/removing video in the middle of a session. For muting and unmuting, the trick is to make judicious use of the “enabled” attribute on the MediaStreamTrack object: simply set a track’s enabled to “false” when you want to mute it.

var pc = new mozRTCPeerConnection;
 
navigator.mozGetUserMedia({video: true},
  function (mediaStream) {
 
    // Create a new self-view video element
    var video = document.createElement("video");
    video.setAttribute("width", 640);
    video.setAttribute("height", 480);
    video.setAttribute("style", "transform: scaleX(-1)");
    video.src = window.URL.createObjectURL(mediaStream);
    document.body.appendChild(video);
    video.play();
 
    // Add a button to hold/unhold video stream
    var button = document.createElement("button");
    button.appendChild(document.createTextNode("Toggle Hold"));
    button.onclick = function(){
      mediaStream.getVideoTracks()[0].enabled =
         !(mediaStream.getVideoTracks()[0].enabled);
    }
    document.body.appendChild(document.createElement("br"));
    document.body.appendChild(button);
 
    // Add the mediaStream to the peer connection
    pc.addStream(mediaStream);
 
    // At this point, you're ready to start the call with
    // pc.setRemoteDescription() or pc.createOffer()
  },
  function (err) { alert(err); }
);

Note that setting a MediaStreamTrack’s “enabled” attribute to “false” will not stop media from flowing, but it will change the media that’s being encoded into a black square (for video) and silence (for audio), both of which compress very well. Depending on your application, it may also make sense to use browser-to-browser signaling (for example, WebSockets or DataChannels) to let the other browser know that it should hide or show the video window when the corresponding video is muted.

Adding video mid-call

For adding video mid-call, the most user-friendly work-around is to destroy the audio-only PeerConnection, and create a new PeerConnection, with both audio and video. When you do this, it will prompt the user for both the camera and the microphone; but, since Firefox does this in a single dialog, the user experience is generally pretty good. Once video has been added, you can either remove it by performing this trick in reverse (thus releasing the camera), or you can simply perform the “mute video” trick I describe above (which will leave the camera going — this might upset some users).

Send more than one audio or video stream

To send more than one audio or video stream, you can use multiple simultaneous peer connections between the browsers: one for each audio/video pair you wish to send. You can also use this technique as an alternate approach for adding and removing video mid-session: set up an initial audio-only call; and, if the user later decides to add video, you can create a new PeerConnection and negotiate a separate video-only connection.

One subtle downside to using the first approach for adding video is that it restarts the audio connection when you add video, which may lead to some noticeable glitches in the audio stream. However, once we have audio and video synchronization landed, making sure that audio and video tracks are in the same MediaStream will ensure that they remain in sync. This synchronization isn’t guaranteed for multiple MediaStreams or multiple PeerConnections.

Temporary workarounds and getting there

We recognize that these workarounds aren’t ideal, and we’re working towards spec compliance as quickly as we can. In the meanwhile, we hope that this information proves helpful in building out applications today. The good news is that these techniques should continue to work even after we’ve addressed the limitations described above, so you can migrate to a final solution at your leisure.

Finally, I would suggest that anyone interested in renegotiation and/or multiple media streams keep a eye on the bugs I mention above. Once we’ve implemented these features, they should appear in the released version of Firefox within about 18 weeks. After that happens, you’ll want to switch over to the “standard” way of doing things to ensure the best possible audio and video quality.

Thanks for your patience. Go out and make great things!

About Adam Roach

Adam Roach works with Mozilla's WebRTC implementation team putting real-time technologies into the core library shared by Firefox and FirefoxOS. He has been crafting the world of Real Time Communications over IP since 1997 by doing protocol standardization, architecture, design, and implementation.

More articles by Adam Roach…

About Robert Nyman [Editor emeritus]

Technical Evangelist & Editor of Mozilla Hacks. Gives talks & blogs about HTML5, JavaScript & the Open Web. Robert is a strong believer in HTML5 and the Open Web and has been working since 1999 with Front End development for the web - in Sweden and in New York City. He regularly also blogs at http://robertnyman.com and loves to travel and meet people.

More articles by Robert Nyman [Editor emeritus]…


9 comments

  1. Kyle Simpson

    Any updates on DataChannels? Specifically:

    1. cross-browser interop?
    2. not needing the “fake” audio stream?
    3. reliable DataChannels?

    September 30th, 2013 at 05:23

  2. Randell Jesup

    1. Cross-browser interop: Chrome has committed their code and are working out a few bugs working with older revs of mozilla; my current belief is that Chrome 31 and Firefox Nightly will interop. I believe Chrome 31 will work with all current versions of Firefox when it comes out.

    2. Firefox hasn’t needed the ‘fake’ audio stream for a while. Firefox 24 does not need it.

    3. Reliable DataChannels have always been fully supported by Firefox (and are the default); the non-standard-compatible DataChannel implementation in current and older Chrome releases is unreliable-only.

    September 30th, 2013 at 08:10

  3. piranna

    Will Firefox support PeerConnection objects from inside WebWorkers, SharedWorkers and ServiceWorkers?

    https://bugzilla.mozilla.org/show_bug.cgi?id=922363

    September 30th, 2013 at 15:15

    1. Robert Nyman [Editor]

      That depends on what will be specified by W3C. There are many things to consider around WebRTC, and we’re evaluating all of it carefully as things progress.

      October 2nd, 2013 at 10:11

    2. Randell Jesup

      See bug 922363 (filed by you); it’s been discussed but would need agreement in the WG to be worth considering. It’s a substantial amount of work (both to spec and to implement).

      October 2nd, 2013 at 10:31

  4. Fabian Gort

    Thanks guys, for the update. How about the ice candidate interop? Is Mozilla going to move the ice candidates in de SDP to the ‘onicecandidate’ event?

    Should we for now just parse the first relevant ice candidate from a FF SDP to create a connection with Chrome?

    October 1st, 2013 at 07:00

    1. Adam Roach

      ICE candidates have interoperated between browsers without any special tricks since our first interop moment many months ago. That said, we do have some behavior variations between Chrome and Firefox that you might be noticing.

      Until Firefox 27, Firefox would not generate trickle ICE candidates, although it would happily accept and properly process the ones that Chrome generates. It did this by ensuring that all of its network information was completely gathered before allowing the initial CreateOffer or SetRemoteDescription operation to be processed.

      Starting with Firefox 27, we will also *generate* trickle ICE candidates, but only when we find out information about new candidates (e.g., from a TURN or STUN server) after the initial SDP handling. Firefox begins gathering its candidates as soon as possible, and includes any that it already knows about in its initial SDP. This means that, under normal network conditions, it’s entirely expected that we might have *all* of our candidates gathered by the time the JavaScript application starts the call. When this happens, all of the candidates are present in the initial offer (or answer), and no trickling will take place.

      By contrast, Chrome’s initial offer or answer — at least, in their current implementation — will never contain any ICE candidates. All address information, including the local interfaces, will be trickled to the application using onicecandidate. This isn’t any more or less correct than what Firefox does; it’s just different. Chrome is also perfectly happy getting all of the ICE candidates in the initial offer (or answer): it doesn’t need them to be trickled in. So it’s just fine with the SDP that Firefox generates, even when it already contains all of our candidates already.

      In either case, as long as your JavaScript application sends the entire initial offer or answer to the other browser, and ensures that subsequent trickle ICE candidates — if any — are also sent, everything should just work. You should never need to parse, synthesize, strip, copy, or modify ICE candidates to get Firefox to interoperate with Chrome.

      One final caveat: early versions of Firefox’s WebRTC implementation did not call onicecandidate at all, even with the “null” candidate that indicates that gathering is complete. While this worked just fine as far as the browsers were concerned, it could sometimes confuse scripts that relied on seeing this callback. This behavior was fixed in Firefox 23. If you had special code to work around this issue, you can safely remove it now. (There’s a regression in Nightly 27 that causes this problem to reappear — Bug 921656 — but it should be fixed very soon).

      If you still think you need a workaround for Chrome interop after the above explanation about how this is supposed to work, please let us know additional details about what unexpected behavior you’re seeing. As far as I know, you shouldn’t need to do anything special.

      October 2nd, 2013 at 09:30

  5. Fabian Gort

    Adam, thanks a lot for your explanation. I wasn’t aware of the ice candidates in the SDP simply being added. Nice to know that it is that simple though.

    I see it works too. That is, with FF stable 24/nightly 27 versus Chrome Canary 31. With FF versus the stable Chrome 29 there I do get the onaddstream event but the stream isn’t becoming visible.

    For me it’s good enough that I got video now between FF and Canary, but if you want to see what I mean, here is an example:
    { “iceServers”: [{ “url”: “stun:stun.l.google.com:19302” }] }

    October 2nd, 2013 at 13:14

  6. Fabian Gort

    sorry, here is the link:
    http://peersquared.info/gvc/test-cam.html

    October 2nd, 2013 at 13:19

Comments are closed for this article.