WebGL Articles

This post is not going to be yet another WebGL tutorial: there already are enough great ones (we list some at the end).

We are just going to introduce the concepts of WebGL, which are basically just the concepts of any general, low-level graphics API (such as OpenGL or Direct3D), to a target audience of Web developers.

What is WebGL?

WebGL is a Web API that allows low-level graphics programming. “Low-level” means that WebGL commands are expressed in terms that map relatively directly to how a GPU (graphics processing unit, i.e. hardware) actually works. That means that WebGL allows you to really tap into the feature set and power of graphics hardware. What native games do with OpenGL or Direct3D, you can probably do with WebGL too.

WebGL is so low-level that it’s not even a “3D” graphics API, properly speaking. Just like your graphics hardware doesn’t really care whether you are doing 2D or 3D graphics, neither does WebGL: 2D and 3D are just two possible usage patterns. When OpenGL 1.0 came out in 1992, it was specifically a 3D API, aiming to expose the features of the 3D graphics hardware of that era. But as graphics hardware evolved towards being more generic and programmable, so did OpenGL. Eventually, OpenGL became so generic that 2D and 3D would be just two possible use cases, while still offering great performance. That was OpenGL 2.0, and WebGL is closely modeled after it.

That’s what we mean when we say that WebGL is a low-level graphics API rather than a 3D API specifically. That is the subject of this article; and that’s what makes WebGL so valuable to learn even if you don’t plan to use it directly. Learning WebGL means learning a little bit of how graphics hardware works. It can help developing an intuition of what’s going to be fast or slow in any graphics API.

The WebGL context and framebuffer

Before we can properly explain anything about the WebGL API, we have to introduce some basic concepts. WebGL is a rendering context for the HTML Canvas element. You start by getting a WebGL context for your canvas:

var gl;
try {
  gl = canvas.getContext("experimental-webgl");
} catch(e) {}

From there, you perform your rendering by calling the WebGL API functions on the gl element obtained there. WebGL is never single-buffered, meaning that the image that you are currently rendering is never the one that is currently displayed in the Canvas element. This ensures that half-rendered frames never show up in the browser’s window. The image being rendered is called the WebGL framebuffer or backbuffer. Talking of framebuffers is made more complicated by the fact that WebGL also allows additional off-screen framebuffers, but let’s ignore that in this article. The image currently being displayed is called the frontbuffer. Of course, the contents of the backbuffer will at some point be copied into the frontbuffer — otherwise WebGL drawing would have no user-visible effect!

But that operation is taken care of automatically by the browser, and in fact, the WebGL programmer has no explicit access to the frontbuffer whatsoever. The key rule here is that the browser may copy the backbuffer into the frontbuffer at any time except during the execution of JavaScript. What this means is that you must perform the entire WebGL rendering of a frame within a single JavaScript callback. As long as you do that, correct rendering is ensured and the browser takes care of the very complex details of multi-buffered compositing for you. You should, in addition, let your WebGL-rendering callback be a requestAnimationFrame callback: if you do so, the browser will also take care of the complex details of animation scheduling for you.

WebGL as a general, low-level graphics API

We haven’t yet described how WebGL is a low-level graphics API where 2D and 3D are just two possible usage patterns. In fact, the very idea that such a general graphics API may exist is non-trivial: it took the industry many years to arrive to such APIs.

WebGL allows to draw either points, or line segments, or triangles. The latter is of course what’s used most of the time, so we will focus entirely on triangles in the rest of this article.

WebGL’s triangle rendering is very general: the application provides a callback, called the pixel shader or fragment shader, that will be called on each pixel of the triangle, and will determine the color in which it should be drawn.

So suppose that you’re coding an old-school 2D game. All what you want is to draw rectangular bitmap images. As WebGL can only draw triangles (more on this below), you decompose your rectange into two triangles as follows,

and your fragment shader, i.e. the program that determines the color of each pixel, is very simple: it will just read one pixel from the bitmap image, and use it as the color for the pixel currently being rendered.

Suppose now that you’re coding a 3D game. You have tesselated your 3D shapes into triangles. Why triangles? Triangles are the most popular 3D drawing primitive because any 3 points in 3D space are the vertices of a triangle. By contrast, you cannot just take any 4 points in 3D space to define a quadrilateral — they would typically fail to lie exactly in the same plane. That’s why WebGL doesn’t care for any other kind of polygons besides triangles.

So your 3D game just needs to be able to render 3D triangles. In 3D, it is a little bit tricky to transform 3D coordinates into actual canvas coordinates — i.e. to determine where in the canvas a given 3D object should end up being drawn. There is no one-size-fits-all formula there: for example, you could want to render fancy underwater or glass refraction effects, that would inevitably require a custom computation for each vertex. So WebGL allows you to provide your own callback, called the vertex shader, that will be called for each vertex of each triangle you will render, and will determine the canvas coordinates at which it should be drawn.

One would naturally expect these canvas coordinates to be 2D coordinates, as the canvas is a 2D surface; but they are actually 3D coordinates, where the Z coordinate is used for depth testing purposes. Two pixels differing only by their Z coordinate correspong to the same pixel on screen, and the Z coordinates are used to determine which one hides the other one. All three axes go from -1.0 to +1.0. It’s important to understand that this is the only coordinate system natively understood by WebGL: any other coordinate system is only understood by your own vertex shader, where you implement the transformation to canvas coordinates.

Once the canvas coordinates of your 3D triangles are known (thanks to your vertex shader), your triangles will be painted, like in the above-discussed 2D example, by your fragment shader. In the case of a 3D game though, your fragment shader will typically be more intricate than in a 2D game, as the effective pixel colors in a 3D game are not as easily determined by static data. Various effects, such as lighting, may play a role in the effective color that a pixel will have on screen. In WebGL, you have to implement all these effects yourself. The good news is that you can: as said above, WebGL lets you specify your own callback, the fragment shader, that determines the effective color of each pixel.

Thus we see how WebGL is a general enough API to encompass the needs of both 2D and 3D applications. By letting you specify arbitrary vertex shaders, it allows implementing arbitrary coordinate transformations, including the complex ones that 3D games need to perform. By accepting arbitrary fragment shaders, it allows implementing arbitrary pixel color computations, including subtle lighting effects as found in 3D games. But the WebGL API isn’t specific to 3D graphics and can be used to implement almost any kind of realtime 2D or 3D graphics — it scales all the way down to 1980s era monochrome bitmap or wireframe games, if that’s what you want. The only thing that’s out of reach of WebGL is the most intensive rendering techniques that require tapping into recently added features of high-end graphics hardware. Even so, the plan is to keep advancing the WebGL feature set as is deemed appropriate to keep the right balance of portability vs features.

The WebGL rendering pipeline

So far we’ve discussed some aspects of how WebGL works, but mostly incidentally. Fortunately, it doesn’t take much more to explain in a systematic way how WebGL rendering proceeds.

The key metaphor here is that of a pipeline. It’s important to understand it because it’s a universal feature of all current graphics hardware, and understanding it will help you instinctively write code that is more hardware-friendly, and thus, runs faster.

GPUs are massively parallel processors, consisting of a large number of computation units designed to work in parallel with each other, and in parallel with the CPU. That is true even in mobile devices. With that in mind, graphics APIs such as WebGL are designed to be inherently friendly to such parallel architectures. On typical work loads, and when correctly used, WebGL allows the GPU to execute graphics commands in parallel with any CPU-side work, i.e. the GPU and the CPU should not have to wait for each other; and WebGL allows the GPU to max out its parallel processing power. It is in order to allow running on the GPU that these shaders are written in a dedicated GPU-friendly language rather than in JavaScript. It is in order to allow the GPU to run many shaders simultaneously that shaders are just callbacks handling one vertex or one pixel each — so that the GPU is free to run shaders on whichever GPU execution unit and in whichever order it pleases.

The following diagram summarizes the WebGL rendering pipeline:

The application sets up its vertex shader and fragment shader, and gives WebGL any data that these shaders will need to read from: vertex data describing the triangles to be drawn, bitmap data (called “textures”) that will be used by the fragment shader. Once this is set up, the rendering starts by executing the vertex shader for each vertex, which determines the canvas coordinates of triangles; the resulting triangles are then rasterized, which means that the list of pixels to be painted is determined; the fragment shader is then executed for each pixel, determining its color; finally, some framebuffer operation determines how this computed color affects the final framebuffer’s pixel color at this location (this final stage is where effects such as depth testing and transparency are implemented).

GPU-side memory vs main memory

Some GPUs, especially on desktop machines, use their own memory that’s separate from main memory. Other GPUs share the same memory as the rest of the system. As a WebGL developer, you can’t know what kind of system you’re running on. But that doesn’t matter, because WebGL forces you to think in terms of dedicated GPU memory.

All what matters from a practical perspective is that:

• WebGL rendering data must first be uploaded to special WebGL data structures. Uploading means copying data from general memory to WebGL-specific memory. These special WebGL data structures are called WebGL textures (bitmap images) and WebGL buffers (generic byte arrays).
• Once that data is uploaded, rendering is really fast.
• But uploading that data is generally slow.

In other words, think of the GPU as a really fast machine, but one that’s really far away. As long as that machine can operate independently, it’s very efficient. But communicating with it from the outside takes very long. So you want to do most of the communication ahead of time, so that most of the rendering can happen independently and fast.

Not all GPUs are actually so isolated from the rest of the system — but WebGL forces you to think in these terms so that your code will run efficiently no matter what particular GPU architecture a given client uses. WebGL data structures abstract the possibility of dedicated GPU memory.

Some things that make graphics slow

Finally, we can draw from what was said above a few general ideas about what can make graphics slow. This is by no means an exhaustive list, but it does cover some of the most usual causes of slowness. The idea is that such knowledge is useful to any programmer ever touching graphics code — regardless of whether they use WebGL. In this sense, learning some concepts around WebGL is useful for much more than just WebGL programming.

Using the CPU for graphics is slow

There is a reason why GPUs are found in all current client systems, and why they are so different from CPUs. To do fast graphics, you really need the parallel processing power of the GPU. Unfortunately, automatically using the GPU in a browser engine is a difficult task. Browser vendors do their best to use the GPU where appropriate, but it’s a hard problem. By using WebGL, you take ownership of this problem for your content.

Having the GPU and the CPU wait for each other is slow

The GPU is designed to be able to run in parallel with the CPU, independently. Inadvertently causing the GPU and CPU to wait for each other is a common cause of slowness. A typical example is reading back the contents of a WebGL framebuffer (the WebGL readPixels function). This may require the CPU to wait for the GPU to finish any queued rendering, and may then also require the GPU to wait for the CPU to have received the data. So as far as you can, think of the WebGL framebuffer as a write-only medium.

Sending data to the GPU may be slow

As mentioned above, GPU memory is abstracted by WebGL data structures such as textures. Such data is best uploaded once to WebGL and then used many times. Uploading new data too frequently is a typical cause of slowness: the uploading is slow by itself, and if you upload data right before rendering with it, the GPU has to wait for the data before it can proceed with rendering — so you’re effectively gating your rendering speed on slow memory transfers.

Small rendering operations are slow

GPUs are intended to be used to draw large batches of triangles at once. If you have 10,000 triangles to draw, doing it in one single operation (as WebGL allows) will be much faster than doing 10,000 separate draw operations of one triangle each. Think of a GPU as a very fast machine with a very long warm-up time. Better warm up once and do a large batch of work, than pay for the warm-up cost many times. Organizing your rendering into large batches does require some thinking, but it’s worth it.

Where to learn WebGL

We intentionally didn’t write a tutorial here because there already exist so many good ones:

• Learning WebGL
• Gregg Tavares’ WebGL articles
• WebGL on MDN

Allow me to also mention that talk I gave, as it has some particularly minimal examples.

This year, Open Web standards like WebGL, Web Workers, Typed Arrays, Fullscreen, and more will have a prominent role in NORAD’s annual mission to track Santa Claus as he makes his journey around the world. That’s because Analytical Graphics, Inc. used Cesium as the basis for the 3D Track Santa application.

Cesium is an open source library that uses JavaScript, WebGL, and other web technologies to render a detailed, dynamic, and interactive virtual globe in a web browser, without the need for a plugin. Terrain and imagery datasets measured in gigabytes or terabytes are streamed to the browser on demand, and overlaid with lines, polygons, placemarks, labels, models, and other features. These features are accurately positioned within the 3D world and can efficiently move and change over time. In short, Cesium brings to the Open Web the kind of responsive, geospatial experience that was uncommon even in bulky desktop applications just a few years ago.

The NORAD Tracks Santa web application goes live on December 24. Cesium, however, is freely available today for commercial and non-commercial use under the Apache 2.0 license.

In this article, I’ll present how Cesium uses cutting edge web APIs to bring an exciting in-browser experience to millions of people on December 24.

The locations used in the screenshots of the NORAD Tracks Santa application are based on test data. We, of course, won’t know Santa’s route until NORAD starts tracking him on Christmas Eve. Also, the code samples in this article are for illustrative purposes and do not necessarily reflect the exact code used in Cesium. If you want to see the official code, check out our GitHub repo.

WebGL

Cesium could not exist without WebGL, the technology that brings hardware-accelerated 3D graphics to the web.

It’s hard to overstate the potential of this technology to bring a whole new class of scientific and entertainment applications to the web; Cesium is just one realization of that potential. With WebGL, we can render scenes like the above, consisting of hundreds of thousands of triangles, at well over 60 frames per second.

Yeah, you could say I’m excited.

If you’re familiar with OpenGL, WebGL will seem very natural to you. To oversimplify a bit, WebGL enables applications to draw shaded triangles really fast. For example, from JavaScript, we execute code like this:

gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer);
 
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, indexBuffer);
 
gl.drawElements(gl.TRIANGLES, numberOfIndices, gl.UNSIGNED_SHORT, 0);

vertexBuffer is a previously-configured data structure holding vertices, or corners of triangles. A simple vertex just specifies the position of the vertex as X, Y, Z coordinates in 3D space. A vertex can have additional attributes, however, such as colors and the vertex’s coordinates within a 2D image for texture mapping.

The indexBuffer links the vertices together into triangles. It is a list of integers where each integer specifies the index of a vertex in the vertexBuffer. Each triplet of indices specifies one triangle. For example, if the first three indices in the list are [0, 2, 1], the first triangle is defined by linking up vertices 0, 2, and 1.

The drawElements call instructs WebGL to draw the triangles defined by the vertex and index buffers. The really cool thing is what happens next.

For every vertex in vertexBuffer, WebGL executes a program, called a vertex shader, that is supplied by the JavaScript code. Then, WebGL figures out which pixels on the screen are “lit up” by each triangle – a process called rasterization. For each of these pixels, called fragments, another program, a fragment shader, is invoked. These programs are written in a C-like language called GLSL that executes on the system’s Graphics Processing Unit (GPU). Thanks to this low-level access and the impressive parallel computation capability of GPUs, these programs can do sophisticated computations very quickly, creating impressive visual effects. This feat is especially impressive when you consider that they are executed hundreds of thousands or millions of times per render frame.

Cesium’s fragment shaders approximate atmospheric scattering, simulate ocean waves, model the reflection of the sun off the ocean surface, and more.

WebGL is well supported in modern browsers on Windows, Linux and Mac OS X. Even Firefox for Android supports WebGL!

While I’ve shown direct WebGL calls in the code above, Cesium is actually built on a renderer that raises the level of abstraction beyond WebGL itself. We never issue drawElements calls directly, but instead create command objects that represent the vertex buffers, index buffers, and other data with which to draw. This allows the renderer to automatically and elegantly solve esoteric rendering problems like the insufficient depth buffer precision for a world the size of Earth. If you’re interested, you can read more about Cesium’s data-driven renderer.

For more information about some of the neat rendering effects used in the NORAD Tracks Santa application, take a look at our blog post on the subject.

Typed Arrays and Cross-Origin Resource Sharing

Virtual globes like Cesium provide a compelling, interactive 3D view of real-world situations by rendering a virtual Earth combined with georeferenced data such as roads, points of interest, weather, satellite orbits, or even the current location of Santa Claus. At the core of a virtual globe is the rendering of the Earth itself, with realistic terrain and satellite imagery.

Terrain describes the shape of the surface: the mountain peaks, the hidden valleys, the wide open plains, and everything in between. Satellite or aerial imagery is then overlaid on this otherwise colorless surface and brings it to life.

The global terrain data used in the NORAD Tracks Santa application is derived from the Shuttle Radar Topography Mission (SRTM), which has a 90-meter spacing between -60 and 60 degrees latitude, and the Global 30 Arc Second Elevation Data Set (GTOPO30), which has 1-kilometer spacing for the entire globe. The total size of the dataset is over 10 gigabytes.

For imagery, we use Bing Maps, who is also a part of the NORAD Tracks Santa team. The total size of this dataset is even bigger – easily in the terabytes.

With such enormous datasets, it is clearly impractical to transfer all of the terrain and imagery to the browser before rendering a scene. For that reason, both datasets are broken up into millions of individual files, called tiles. As Santa flies around the world, Cesium downloads new terrain and imagery tiles as they are needed.

Terrain tiles describing the shape of the Earth’s surface are binary data encoded in a straightforward format. When Cesium determines that it needs a terrain tile, we download it using XMLHttpRequest and access the binary data using typed arrays:

var tile = ...
 
var xhr = new XMLHttpRequest();
 
xhr.open('GET', terrainTileUrl, true);
 
xhr.responseType = 'arraybuffer';
 
 
 
xhr.onload = function(e) {
 
    if (xhr.status === 200) {
 
        var tileData = xhr.response;
 
        tile.heights = new Uint16Array(tileData, 0, heightmapWidth * heightmapHeight);
 
        var heightsBytes = tile.heights.byteLength;
 
        tile.childTileBits = new Uint8Array(tileData, heightsBytes, 1)[0];
 
        tile.waterMask = new Uint8Array(tileData, heightsBytes + 1, tileData.byteLength - heightsBytes - 1);
 
        tile.state = TileState.RECEIVED;
 
    } else {
 
        // ...
 
    }
 
};
 
 
 
xhr.send();

Prior to the availability of typed arrays, this process would have been much more difficult. The usual course was to encode the data as text in JSON or XML format. Not only would such data be larger when sent over the wire(less), it would also be significantly slower to process it once it was received.

While it is generally very straightforward to work with terrain data using typed arrays, two issues make it a bit trickier.

The first is cross-origin restrictions. It is very common for terrain and imagery to be hosted on different servers than are used to host the web application itself, and this is certainly the case in NORAD Tracks Santa. XMLHttpRequest, however, does not usually allow requests to non-origin hosts. The common workaround of using script tags instead of XMLHttpRequest won’t work well here because we are downloading binary data – we can’t use typed arrays with JSONP.

Fortunately, modern browsers offer a solution to this problem by honoring Cross-Origin Resource Sharing (CORS) headers, included in the response by the server, indicating that the response is safe for use across hosts. Enabling CORS is easy to do if you have control over the web server, and Bing Maps already includes the necessary headers on their tile files. Other terrain and imagery sources that we’d like to use in Cesium are not always so forward-thinking, however, so we’ve sometimes been forced to route cross-origin requests through a same-origin proxy.

The other tricky aspect is that modern browsers only allow up to six simultaneous connections to a given host. If we simply created a new XMLHttpRequest for each tile requested by Cesium, the number of queued requests would grow large very quickly. By the time a tile was finally downloaded, the viewer’s position in the 3D world may have changed so that the tile is no longer even needed.

Instead, we manually limit ourselves to six outstanding requests per host. If all six slots are taken, we won’t start a new request. Instead, we’ll wait until next render frame and try again. By then, the highest priority tile may be different than it was last frame, and we’ll be glad we didn’t queue up the request then. One nice feature of Bing Maps is that it serves the same tiles from multiple hostnames, which allows us to have more outstanding requests at once and to get the imagery into the application faster.

Web Workers

The terrain data served to the browser is, primarily, just an array of terrain heights. In order to render it, we need to turn the terrain tile into a triangle mesh with a vertex and index buffer. This process involves converting longitude, latitude, and height to X, Y, and Z coordinates mapped to the surface of the WGS84 ellipsoid. Doing this once is pretty fast, but doing it for each height sample, of which each tile has thousands, starts to take some measurable time. If we did this conversion for several tiles in a single render frame, we’d definitely start to see some stuttering in the rendering.

One solution is to throttle tile conversion, doing at most N per render frame. While this would help with the stuttering, it doesn’t avoid the fact that tile conversion competes with rendering for CPU time while other CPU cores sit idle.

Fortunately, another great new web API comes to the rescue: Web Workers.

We pass the terrain ArrayBuffer downloaded from the remote server via XMLHttpRequest to a Web Worker as a transferable object. When the worker receives the message, it builds a new typed array with the vertex data in a form ready to be passed straight to WebGL. Unfortunately, Web Workers are not yet allowed to invoke WebGL, so we can’t create vertex and index buffers in the Web Worker; instead, we post the typed array back to the main thread, again as a transferable object.

The beauty of this approach is that terrain data conversion happens asynchronously with rendering, and that it can take advantage of the client system’s multiple cores, if available. This leads to a smoother, more interactive Santa tracking experience.

Web Workers are simple and elegant, but that simplicity presents some challenges for an engine like Cesium, which is designed to be useful in various different types of applications.

During development, we like to keep each class in a separate .js file, for ease of navigation and to avoid the need for a time-consuming combine step after every change. Each class is actually a separate module, and we use the Asynchronous Module Definition (AMD) API and RequireJS to manage dependencies between modules at runtime.

For use in production environments, it is a big performance win to combine the hundreds of individual files that make up a Cesium application into a single file. This may be a single file for all of Cesium or a user-selected subset. It may also be beneficial to combine parts of Cesium into a larger file containing application-specific code, as we’ve done in the NORAD Tracks Santa application. Cesium supports all of these use-cases, but the interaction with Web Workers gets tricky.

When an application creates a Web Worker, it provides to the Web Worker API the URL of the .js file to invoke. The problem is, in Cesium’s case, that URL varies depending on which of the above use-cases is currently in play. Worse, the worker code itself needs to work a little differently depending on how Cesium is being used. That’s a big problem, because workers can’t access any information in the main thread unless that information is explicitly posted to it.

Our solution is the cesiumWorkerBootstrapper. Regardless of what the WebWorker will eventually do, it is always constructed with cesiumWorkerBootstrapper.js as its entry point. The URL of the bootstrapper is deduced by the main thread where possible, and can be overridden by user code when necessary. Then, we post a message to the worker with details about how to actually dispatch work.

var worker = new Worker(getBootstrapperUrl());
 
 
 
//bootstrap
 
var bootstrapMessage = {
 
    loaderConfig : {},
 
    workerModule : 'Workers/' + processor._workerName
 
};
 
 
 
if (typeof require.toUrl !== 'undefined') {
 
    bootstrapMessage.loaderConfig.baseUrl = '..';
 
} else {
 
    bootstrapMessage.loaderConfig.paths = {
 
        'Workers' : '.'
 
    };
 
}
 
worker.postMessage(bootstrapMessage);

The worker bootstrapper contains a simple onmessage handler:

self.onmessage = function(event) {
 
    var data = event.data;
 
    require(data.loaderConfig, [data.workerModule], function(workerModule) {
 
        //replace onmessage with the required-in workerModule
 
        self.onmessage = workerModule;
 
    });
 
};

When the bootstrapper receives the bootstrapMessage, it uses the RequireJS implementation of require, which is also included in cesiumWorkerBootstrapper.js, to load the worker module specified in the message. It then “becomes” the new worker by replacing its onmessage handler with the required-in one.

In use-cases where Cesium itself is combined into a single .js file, we also combine each worker into its own .js file, complete with all of its dependencies. This ensures that each worker needs to load only two .js files: the bootstrapper plus the combined module.

Mobile Devices

One of the most exciting aspects of building an application like NORAD Tracks Santa on web technologies is the possibility of achieving portability across operating systems and devices with a single code base. All of the technologies used by Cesium are already well supported on Windows, Linux, and Mac OS X on desktops and laptops. Increasingly, however, these technologies are becoming available on mobile devices.

The most stable implementation of WebGL on phones and tablets is currently found in Firefox for Android. We tried out Cesium on several devices, including a Nexus 4 phone and a Nexus 7 tablet, both running Android 4.2.1 and Firefox 17.0. With a few tweaks, we were able to get Cesium running, and the performance was surprisingly good.

We did encounter a few problems, however, presumably a result of driver bugs. One problem was that normalizing vectors in fragment shaders sometimes simply does not work. For example, GLSL code like this:

vec3 normalized = normalize(someVector);

Sometimes results in a normalized vector that still has a length greater than 1. Fortunately, this is easy to work around by adding another call to normalize:

vec3 normalized = normalize(normalize(someVector));

We hope that as WebGL gains more widespread adoption on mobile, bugs like this will be detected by the WebGL conformance tests before devices and drivers are released.

The Finished Application

As long-time C++ developers, we were initially skeptical of building a virtual globe application on the Open Web. Would we be able to do all the things expected of such an application? Would the performance be good?

I’m pleased to say that we’ve been converted. Modern web APIs like WebGL, Web Workers, and Typed Arrays, along with the continual and impressive gains in JavaScript performance, have made the web a convenient, high-performance platform for sophisticated 3D applications. We’re looking forward to continuing to use Cesium to push the limits of what is possible in a browser, and to take advantage of new APIs and capabilities as they become available.

We’re also looking forward to using this technology to bring a fun, 3D Santa tracking experience to millions of kids worldwide this Christmas as part of the NORAD Tracks Santa team. Check it out on December 24 at www.noradsanta.org.

Some good reading about an Open SWF Runtime Project, WebRTC photobooth, Wikimedia HTML5 Media Player with WebM Support, WebGL/GLSL plasma simulation running on the GPU and more in store for you! Here are this week’s link tips from Mozilla’s Developer Engagement team.

Continued…

Mozilla has been rolling out two new resources for developers to hack on the network of the future:

• Learning Labs to get hackers started using current web technologies such as WebGL or WebRTC on networks unconstrained by bandwidth, latency, or compute capacity.
• Developer Docs so you can quickly start hacking on the next generation of the internet — “software defined networking” capable networks.

These SDN networks are deeply programmable as well as ultra high speed. They allow you to program the entire network, not just the edge devices — define the network topology, route packets however you want, and put your code through the entire network from edge to router to firewall. Happy hacking!

Learning Labs

Each “learning lab” is a short video and sample code that lets you explore an emerging technology on unconstrained networks.

The first out of the forge focuses on WebGL, the new JavaScript API for hardware accelerated graphics right inside the browser. We interviewed folks from Google and TeamUp and individual hackers like Andor Salga about WebGL’s transformative potential in education, medicine, and in-browser graphics in general. We also point you to great WebGL resources to level up your skills more generally.

Get your hands dirty with a hackable demo in node.js to see WebGL in action. The app streams a 3D point cloud from server to client and renders it right in the browser. Take a look at see how ultra high speed networks can have a dramatic impact on what’s possible with real world applications.

There are apps that become possible on these ultra high speed, programmable networks that are simply not possible on today’s internet.

Here’s a quick screencast of the project in action.

When you’re done with the experiment, jump into the source code see how it all works, fork it, make tweaks, and use it as inspiration for your killer app of the future.

Developer Docs for Programmable Networks

If you’re writing an app for the future, you’ll probably want to run it on a network of the future, right? That’s where GENI comes in.

GENI is a project sponsored by the National Science Foundation and built in collaboration by a number of academic institutions and researchers. It’s a network and testbed that supports ultra high speeds and Software Defined Networking. We’ve written some awesome docs to get you started so you can get up and running on GENI with your killer app of the future. You can also find them on the Ignite resource page.

The goal of creating these technologies is to make apps with major impact on peoples’ lives that are not possible on today’s internet.

Get involved in Mozilla Ignite.

The Mozilla Ignite Challenge is a project to do just that — create apps with impact in areas of public benefit like health, public safety, education and workforce development and others, using these next generation networking technologies.

Get involved in the Mozilla Ignite community by emailing ignite [at] mozillafoundation [dot] org to learn more.

Stay tuned for future labs. We’ll explore WebRTC, Websockets/Socket.io, SPDY, AppCache, file access API, and more in upcoming learning labs.

These technologies combined with ultra high-speed, programmable networking are going to have a major impact on what we can do with the internet going forward.

As always, we welcome your feedback at ignite [at] mozillafoundation [dot] org!

The web moves on and interesting things pop up! Here are this week’s link from Mozilla’s Developer Engagement team.

Continued…

Paul Brunt won the WebGL Dev Derby with SnappyTree, his incredibly powerful (and even a little addicting) 3D tree designer. SnappyTree provides a wonderful example of what we can do with the Web today — it even has an export function for using your trees in native applications (move over Blender), not that we will need native 3D applications if progress like this continues.

Recently, I had the opportunity to learn more about Paul and his work. In our interview, Paul shared thoughts on the past, present, and future of web development and gave advice relevant to developers of all levels of experience.

The Interview

How did you become interested in web development?

I first became interested in web development in the late 90′s when I went on the internet for the first time. After a little bit of browsing around the web I thought I’d have a go at making a site myself. With the free space provided by the ISP and with a little assistance from Frontpage I managed to put together something you might tentatively call a website. Unfortunately that first site was short lived, the result of an accidental deletion. But, the seed had been planted so I built another, then another, then somewhere along the way I ended up doing it for a living.

Tell us about developing SnappyTree. Was anything especially exciting, challenging, or rewarding?

It had taken a few days of coding before I was in the position to start drawing anything to the screen. So, the most exciting part was seeing something that resembled a tree appear in the browser for the first time; although it wasn’t much to look at, and there were very obviously issues.

Getting skinning working correctly was extremely challenging; there where many horrible branch twisted iterations while trying to get it right, but I’m pretty happy with the end result.

Can you tell us a little about how SnappyTree works?

Snappy Tree works by taking an initial branch (the trunk) which then splits into two new branches. The direction of these new branches is determined by several user configurable factors symmetry, droopyness, etc. This process is repeated N times to produce the basic tree structure.

After the basic tree has been constructed a skin in generated around the branches before finally adding planes at the ends of the branches which are used for leaves and twigs. The final mesh data is then piped into Webgl for rendering and used to generate the collada or wavefront files for export.

What makes the web an exciting platform for you?

Increasingly it seems to be getting quicker and easier to develop for the web than any native platform, so I think the current rate of progress is the most exciting part. With so many new technologies emerging seeing how developers use them is always fun and often downright awe inspiring.

What up-and-coming web technologies are you most excited about?

I think the most exciting thing emerging right now is WebRTC. I’m really looking forward to seeing what uses developers can come up with. I can see a lot of potential outside of the obvious and it’s going the be a lot of fun discovering interesting uses for it.

If you could change one thing about the web, what would it be?

There are lots of little things I’d love to tweak in CSS and HTML; but, if it’s limited to just one thing I think I would change the “www.” convention. It’s difficult to pronounce, takes far too long to say and just sounds horrible.

What advice would you give to aspiring web developers?

Jump in the deep end and be ambitious. The best way to learn a new technology is to start a project around that technology. Even if you haven’t got a clue what you’re doing when you start you certainly will by the end.

Further Reading

• Mozilla and Games: Pushing the Limits of What’s Possible
• WebRTC efforts underway at Mozilla!
• The Great Dub-Dub-Dub Debate

Last month, creative web developers from around the world showed us what they could do with WebGL in the June Dev Derby competition. After looking through the entries, our three expert judges—Guillermo Rauch, Peter Lubbers, and Rob Hawkes—have decided on three winners and two runners-up.

Not a contestant? There are other reasons to be excited. Most importantly, all of the entries are completely open-source, making them wonderful lessons in all of the exciting things you can do with WebGL today.

Winners

• SnappyTree by newcomer Paul Brunt
• Slice:Drop by newcomer Daniel Haehn
• Kai ‘Opua by newcomer Collin Hover

Runners-up

• 3d Grapher by sophomore guska076
• FLUID by veteran Emeric “boblemarin” Florence

Relatively few demos were submitted this time around, but the quality of the entries was outstanding. Each and every finalist has created something truly amazing, something that pushes WebGL forward in an inspiring way. And let’s not forget about the other jaw-dropping entries. Games, planets, maps, mirrors, and more — every entry is extremely impressive. Congratulations to all of our contestants!

Want to get a head start on an upcoming Derby? We are now accepting demos for the No JavaScript Challenge (July) and demos related to the Camera API (August). Get started today!

Further Reading

• Getting started with WebGL
• More MDN documentation on WebGL
• Learning WebGL tutorials

This is the second of a series of articles dedicated to the useful libraries that all web developers should have in their toolbox. The intent is to show you what those libraries can do and help you to use them at their best. This second article is dedicated to the ThreeJS library.

Introduction

ThreeJS is a library originally written by Ricardo Cabello Miguel aka “Mr. Doob“.

This library makes WebGL accessible to common human beings. WebGL is a powerful API to manipulate 3D environments. This web technology is standardized by the Kronos group and Firefox, Chrome and Opera now implement it as a 3D context for the HTML canvas tag. WebGL is basically a web version of another standard : OpenGL ES 2.0. As a consequence, this API is a “low level” API that require skills and knowledge beyond what web designers are used to. That’s where ThreeJS comes into play. ThreeJS gives web developers access to the power of WebGL without all the knowledge required by the underlying API.

Basic usage

The library has good documentation with many examples. You’ll notice that some parts of the documentation are not complete yet (feel free to help). However, the library and examples source code are very well structured, so do not hesitate to read the source.

Even though ThreeJS simplifies many things, you still have to be comfortable with some basic 3D concepts. Basically, ThreeJS uses the following concepts:

1. The scene: the place where all 3D objects will be placed and manipulated in a 3D space.
2. The camera: a special 3D object that will define the rendering point of view as well as the type of spatial rendering (perspective or isometric)
3. The renderer: the object in charge of using the scene and the camera to render your 3D image.

Within the scene, you will have several 3D objects which can be of the following types:

• A mesh: A mesh is an object made of a geometry (the shape of your object) and a material (its colors and texture)
• A light point: A special object that defines a light source to highlight all your meshes.
• A camera, as described above.

The following example will draw a simple wireframe sphere inside an HTML element with the id “myPlanet”.

/**
 * First, let's prepare some context
 */
 
// The WIDTH of the scene to render
var __WIDTH__  = 400,
 
// The HEIGHT of the scene to render
    __HEIGHT__ = 400,
 
// The angle of the camera that will show the scene
// It is expressed in degrees
    __ANGLE__ = 45,
 
// The shortest distance the camera can see
    __NEAR__  = 1,
 
// The farthest distance the camera can see
    __FAR__   = 1000
 
// The basic hue used to color our object
    __HUE__   = 0;
 
/**
 * To render a 3D scene, ThreeJS needs 3 elements:
 * A scene where to put all the objects
 * A camera to manage the point of view
 * A renderer place to show the result
 */
var scene  = new THREE.Scene(), 
    camera = new THREE.PerspectiveCamera(__ANGLE__, 
                                         __WIDTH__ / __HEIGHT__, 
                                         __NEAR__, 
                                         __FAR__),
    renderer = new THREE.WebGLRenderer();
 
/**
 * Let's prepare the scene
 */
 
// Add the camera to the scene
scene.add(camera);
 
// As all objects, the camera is put at the 
// 0,0,0 coordinate, let's pull it back a little
camera.position.z = 300;
 
// We need to define the size of the renderer
renderer.setSize(__WIDTH__, __HEIGHT__);
 
// Let's attach our rendering zone to our page
document.getElementById("myPlanet").appendChild(renderer.domElement);
 
/**
 * Now we are ready, we can start building our sphere
 * To do this, we need a mesh defined with:
 *  1. A geometry (a sphere) 
 *  2. A material (a color that reacts to light)
 */
var geometry, material, mesh;
 
// First let's build our geometry
//
// There are other parameters, but you basically just 
// need to define the radius of the sphere and the 
// number of its vertical and horizontal divisions.
//
// The 2 last parameters determine the number of 
// vertices that will be produced: The more vertices you use, 
// the smoother the form; but it will be slower to render. 
// Make a wise choice to balance the two.
geometry = new THREE.SphereGeometry( 100, 20, 20 );
 
// Then, prepare our material
var myMaterial = {
    wireframe : true,
    wireframeLinewidth : 2
}
 
// We just have to build the material now
material = new THREE.MeshPhongMaterial( myMaterial );
 
// Add some color to the material
material.color.setHSV(__HUE__, 1, 1);
 
// And we can build our the mesh
mesh = new THREE.Mesh( geometry, material );
 
// Let's add the mesh to the scene
scene.add( mesh );
 
/**
 * To be sure that we will see something, 
 * we need to add some light to the scene
 */
 
// Let's create a point light
var pointLight = new THREE.PointLight(0xFFFFFF);
 
// and set its position
pointLight.position.x = -100;
pointLight.position.y = 100;
pointLight.position.z = 400;
 
// Now, we can add it to the scene
scene.add( pointLight );
 
 
// And finally, it's time to see the result
renderer.render( scene, camera );

And if you want to animate it (for example, make the sphere spin), it’s this easy:

function animate() {
    // beware, you'll maybe need a shim 
    // to use requestAnimationFrame properly
    requestAnimationFrame( animate );
 
    // First, rotate the sphere
    mesh.rotation.y -= 0.003;
 
    // Then render the scene
    renderer.render( scene, camera );
}
 
animate();

JSFiddle demo.

Advanced usage

Once you master the basics, ThreeJS provides you with some advanced tools.

Rendering system

As an abstraction layer, ThreeJS offer options to render a scene other than with WebGL. You can use the Canvas 2D API as well as SVG to perform your rendering. There is some difference between all these rendering contexts. The most obvious one is performance. Because WebGL is hardware accelerated, the rendering of complex scene is amazingly faster with it. On the other hand, because WebGL does not deal always well with anti-aliasing, the SVG or Canvas2D rendering can be better if you want to perform some cell-shading (cartoon-like) stuff. As a special advantage, SVG rendering gives you a full DOM tree of objects, which can be useful if you want access to those objects. It can have a high cost in term of performance (especially if you animate your scene), but it allows you to not rebuild a full retained mode graphic API.

Mesh and particles

ThreeJS is perfect for rendering on top of WebGL, but it is not an authoring tool. To model 3D objects, you have a choice of 3D software. Conveniently, ThreeJS is available with many scripts that make it easy to import meshes from several sources (Examples include: Blender, 3DSMax or the widely supported OBJ format).

It’s also possible to easily deploy particle systems as well as using Fog, Matrix and custom shaders. ThreeJS also comes with a few pre-built materials: Basic, Face, Lambert, Normal, and Phong). A WebGL developer will be able to build his own on top of the library, which provide some really good helpers. Obviously, building such custom things requires really specific skills.

Animating mesh

If using requestAnimationFrame is the easiest way to animate a scene, ThreeJS provides a couple of useful tools to animate meshes individually: a full API to define how to animate a mesh and the ability to use “bones” to morph and change a mesh.

Limits and precaution

One of the biggest limitations of ThreeJS is related to WebGL. If you want to use it to render your scene, you are constrained by the limitations of this technology. You become hardware dependent. All browsers that claim to support WebGL have strong requirements in terms of hardware support. Some browsers will not render anything if they do not run with an appropriate hardware. The best way to avoid trouble is to use a library such as modernizr to switch between rendering systems based on each browser’s capabilities. However, take care when using non-WebGL rendering systems because they are limited (e.g. the Phong material is only supported in a WebGL context) and infinitely slower.

In terms of browser support, ThreeJS supports all browsers that support WebGL, Canvas2D or SVG, which means: Firefox 3.6+, Chrome 9+, Opera 11+, Safari 5+ and even Internet Explorer 9+ if you do not use the WebGL rendering mode. If you want to rely on WebGL, the support is more limited: Firefox 4+, Chrome 9+, Opera 12+, Safari 5.1+ only. You can forget Internet Explorer (even the upcoming IE10) and almost all mobile browsers currently available.

Conclusion

ThreeJS drastically simplifies the process of producing 3D images directly in the browser. It gives the ability to do amazing visual effects with an easy to use API. By empowering you, it allow you to unleash your creativity.

In conclusion, here are some cool usages of ThreeJS:

• http://highrise.nfb.ca/onemillionthtower/1mt_webgl.php
• http://www.thewildernessdowntown.com/ (To watch this one, check that your browser is allowed to move and resize pop-up windows.)
• http://www.playmapscube.com/
• http://www.ro.me

As reported by Creative JS, Jasmine Kent ported her open source Linux racing game Trigger Rally to WebGL to run in HTML5 browsers. In the following video you can see it running pretty smoothly on my MacBook air that is already running 12% of CPU without it starting (as ScreenFlow is recording and VLC was playing music whilst TweetDeck used quite a chunk, too):

If you are the artistic type, you can even design your own levels for the game.

You can play play Trigger Rally online here: http://triggerrally.com/ or follow it on Twitter @triggerrally.

We’re happy to announce the availability of Aurora 10.
(Download and Test Aurora 10)

In additional to the normal improvements that you’ve come to expect like performance, security and bug fixes, Aurora 10 focuses in HTML5 enhancements.

New additions

• HTML5 Visibility API
• createProcessingInstruction
• WebGL antialiasing
• 3D Transforms
• Visibility API
• Document.mozFullScreenEnabled\

Developer Tools

• Code Editor
• CSS Rules View
• Page Inspector/Highlighter
• Console Object Completion
• Style Inspector

Aurora 10 also implements incremental enhancements like IndexedDB setVersion API changes. Ongoing detailed attention to evolving specifications help to keep Firefox at the front of the Web revolution. (Read more about IndexedDB on MDN.)

DOM

• We now fire a “load” event on stylesheet linking when the sheet load finishes or “error” if the load fails.
• We turn the POSTDATA prompt into an information page (when navigating in session history).
• We only forward event attributes on body/frameset to the window if we also forward the corresponding on* property.
• We no longer allow more than one call to window.open() when we allow popups.
• We fixed a bug where a success callback never fired when a position update is triggered after getCurrentPosition().
• We removed replaceWholeText().
• We fixed an error with createPattern(zero-size canvas).
• We now handle putImageData(nonfinite) correctly.
• We now throw INVALID_STATE_ERR when dispatching uninitialized events.
• We’ve made Document.documentURI readonly.
• We fixed document.importNode to comply with optional argument omitted.

Web workers

• We now allow data URLs.
• We implemented event.stopImmediatePropagation in workers.
• We made XHR2 response/responseType work in Web Workers.

Graphics

• We implement the WebGL OES_standard_derivatives extension.
• We implement minimal-capabilities WebGL mode.

JavaScript

• The function caller property no longer skips over eval frames.
• We fixed E4X syntax so that it is not accepted in ES5 strict mode.
• weakmap.set no longer returns itself instead of undefined.
• We implemented the battery API.

Offline: IndexedDB enhancements

• IndexedDB setVersion API changes
• Added support for IDBObjectStore/IDBIndex.count
• Various methods accept both keys and KeyRanges.
• Added support for IDBCursor.advance.
• Implemented deleteDatabase.
• objectStoreNames are no longer updated on closed databases when another connection adds or removes object stores
• IDBObjectStore.delete and IDBCursor.delete now return undefined.
• No longer throws an error if there are unknown properties in the options objects to createObjectStore/createIndex.
• We now the errorCode to “ABORT_ERR” for all pending requests when IDBTransaction.abort() is called.
• Fixed the sort order for indexes.

Layout

• We have updated the current rule for handling malformed media queries.
• We now support the HTML5 <bdi> element and CSS property unicode-bidi: isolate.
• The CSS3 implementation now supports unicode-bidi: plaintext.

Media

• Implemented Document.mozFullScreenEnabled.
• Enabled the DOM full-screen API on desktop Firefox by default.