WebGL Off the Main Thread

We’re happy to announce WebGL in Web Workers in Firefox 44+! Using the new OffscreenCanvas API you can now create a WebGL context off of the main thread.

To follow along, you’ll need a copy of Firefox 44 or newer (currently Firefox Developer Edition or Firefox Nightly). You’ll have to enable this API by navigating to about:config in Firefox, searching for gfx.offscreencanvas.enabled, and setting it to true. You can grab the code examples from GitHub or preview it here in Firefox 44+ with gfx.offscreencanvas.enabled set to true. This functionality is not yet available on Windows pending ANGLE support. AlteredQualia points out things are running great on Windows/FF Nightly 46. Shame on me for not verifying!

Use Cases

This API is the first that allows a thread other than the main thread to change what is displayed to the user. This allows rendering to progress no matter what is going on in the main thread. You can see more use cases in the working group’s in-progress specification.

Code Changes

Let’s take a look at a basic example of WebGL animation from my Raw WebGL talk. We’ll port this code to run in a worker, rather than the main thread.

WebGL in Workers

The first step is moving all of the code from WebGL context creation to draw calls into a separate file.


<script src="gl-matrix.js"></script>
<script>
  // main thread
  var canvas = document.getElementById('myCanvas');
  ...
  gl.useProgram(program);
  ...

becomes:


// main thread
var canvas = document.getElementById('myCanvas');
if (!('transferControlToOffscreen' in canvas)) {
  throw new Error('webgl in worker unsupported');
}
var offscreen = canvas.transferControlToOffscreen();
var worker = new Worker('worker.js');
worker.postMessage({ canvas: offscreen }, [offscreen]);
...

Recognize that we’re calling HTMLCanvasElement.prototype.transferControlToOffscreen, then transferring that to a newly constructed worker thread. transferControlToOffscreen returns a new object which is an instance of OffscreenCanvas, as opposed to HTMLCanvasElement. While similar, you can’t access properties like offscreen.clientWidth and offscreen.clientHeight, but you can access offscreen.width and offscreen.height. By passing it as the second argument to postMessage, we transfer ownership of the variable to the second thread.

Now in the worker thread, we’ll wait to receive the message from the main thread with the canvas element, before trying to get a WebGL context. The code for getting a WebGL context, creating and filling buffers, getting and setting attributes and uniforms, and drawing does not change.


// worker thread
importScripts('gl-matrix.js');

onmessage = function (e) {
  if (e.data.canvas) {
    createContext(e.data.canvas);
  }
};

function createContext (canvas) {
  var gl = canvas.getContext('webgl');
  ...

OffScreenCanvas simply adds one new method to WebGLRenderingContext.prototype called commit. The commit method will push the rendered image to the canvas element that created the OffscreenCanvas used by the WebGL context.

Animation Synchronization

Now to get the code animating, we can proxy requestAnimationFrame timings from the main thread to the worker with postMessage.


// main thread
(function tick (t) {
  worker.postMessage({ rAF: t });
  requestAnimationFrame(tick);
})(performance.now());

and onmessage in the worker becomes:


// worker thread
onmessage = function (e) {
  if (e.data.rAF && render) {
    render(e.data.rAF);
  } else if (e.data.canvas) {
    createContext(e.data.canvas);
  }
};

and our render function now has a final gl.commit(); statement rather than setting up another requestAnimationFrame loop.


// main thread
function render (dt) {
  // update
  ...
  // render
  gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);
  gl.drawArrays(gl.TRIANGLES, 0, n);
  requestAnimationFrame(render);
};

becomes:


// worker thread
function render (dt) {
  // update
  ...
  // render
  gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);
  gl.drawArrays(gl.TRIANGLES, 0, n);
  gl.commit(); // new for webgl in workers
};

Limitations with this approach

While in the example code, I’m not doing proper velocity-based animation (by not using the value passed from requestAnimationFrame, I’m doing frame rate dependate animation, as opposed to the more correct velocity based animation which is frame rate independent), we still have an issue with this approach.

Assume we moved the rendering logic off of the main thread to avoid pauses from the JavaScript Virtual Machine’s Garbage Collector (GC pauses). GC pauses on the main thread will slow down invocations of requestAnimationFrame. Since calls to gl.drawArrays and gl.commit are asynchronously triggered in the worker thread by postMessages in a requestAnimationFrame loop on the main thread, GC pauses in the main thread will block rendering on the worker thread. Note: GC pauses in the main thread should not block progress in a worker thread (at least they don’t in Firefox’s SpiderMonkey Virtual Machine). GC pauses are per Worker in SpiderMonkey.

While we could try to do something clever in the worker to account for this, the solution will be to make requestAnimationFrame available in a Worker context. The bug tracking this work can be found here.

Summary

Developers will now be able to render to the screen without blocking on the main thread, thanks to the new OffscreenCanvas API. There’s still more work to do with getting requestAnimationFrame on Workers. I was able to port existing WebGL code to run in a worker in a few minutes. For comparison, see animation.html vs. animation-worker.html and worker.js.

About Nick Desaulniers

Nick Desaulniers is a software engineer fighting for the open web at Mozilla. When Nick is not helping third party developers target Firefox OS, he’s giving talks about graphics programming with WebGL, contributing to Open Source, or compiling C/C++ to JavaScript with Emscripten. Nick is fond of learning numerous programming languages, just in time compilation, high performance JavaScript, and graphics programming. Contributing to Open Source software and an accessible Internet for all are some of the things that Nick is most passionate about.

More articles by Nick Desaulniers…


6 comments

  1. Brian Gavin

    This could be a useful feature. Are other Web Browsers looking at doing this?

    January 24th, 2016 at 05:02

  2. Zimondai

    This is the feature I am waiting for!!!!!
    Hope it would gain support from other browsers soon.

    Currently I uses a class providing all Canvas methods and properties to simulate `Canvas` in web worker to generate a command list array, which will be sent to the main thread for actually painting. It’s dirty but works fine.

    January 24th, 2016 at 17:45

  3. Morris Tseng

    ANGLE has problem when there are multiple workers using OffscreenCanvas simultaneously (The program will crash). If there is only one worker, there is no problem.

    January 27th, 2016 at 19:44

  4. Gordon Rankin

    This is fantastic and i’m very excited to start playing with it. But i’m left wondering how we are supposed to manage interactivity? Currently we are using a scene graph in the main thread built up of many javascript objects. Mouse/touch events directly manipulate the positions and size of the objects, as well as change various shader uniforms etc. Our render passes through them each frame, batches them where possible and draws them, be they sprites, primitives or whatever.

    Using the Offscreen method should we continue to manage all our rendereable objects (positions, scale, size etc) in the main thread, manipulate them as we do now via mouse/touch events, and then simply proxy up all webgl commands and send them to the worker?

    Or are we supposed to keep our whole javascript scene and object classes in the webworker itself and proxy into the worker the mouse and touch events to manipulate the scene and its objects?

    I’m loving the sound of this, but I feel a little lost on how we should go about harnessing it for large complex games/apps that require interactivity and make heavy use of a scene graph.

    January 28th, 2016 at 08:27

  5. Niels Rood

    @Gordon Rankin I have not yet tried this out, but I think it would be best to move as much of the rendering code to the Worker, reducing the load on the main thread. Therefore I would pass the information needed from the Mouse and Touch events directly to the Worker through postMessage. It is probably possible to reduce the postMessage overhead by using Transferable objects.

    Very nice piece of technology! Looking forward to the time when this is available across all browsers :)

    January 29th, 2016 at 03:10

  6. Alex Bell

    Incredibly powerful feature here, thank you. Is there a specification for this? Is there a Chromium bug/interest expressed? Is there a FF timetable for exposing by default?

    February 8th, 2016 at 12:49

Comments are closed for this article.