Mozilla

Monster Madness – creating games on the web with Emscripten

When our engineering teams at Trendy Entertainment & Nom Nom Games decided on the strategy of developing one of our new Unreal Engine 3 games — Monster Madness Online — as a cross-platform title, we knew that a frictionless multiplayer web browser version would be central to this experience. The big question, however, was determining what essential technologies to utilize in order to bring our game onto the web. As a C++ oriented developer, we determined quickly that rewriting the game engine from the ground-up was out of the question. We’d need a solution that would allow us to port our existing code in an efficient manner into a format usable in the browser…

TL;DR? Watch the video!

Playing the Field

We looked hard at the various options in front of us: FlasCC (a GCC Flash compiler), Google’s NaCl, a custom native C++ extension, or Mozilla’s Emscripten & asm.js.

In our tests, Flash ran slowly and had inconsistent behaviors between Pepper (Chrome) and Adobe’s plugin version. Combined with the increasingly onerous plugin requirement, we opted to look elsewhere for a more seamless, forward-thinking approach.

NaCl had the issue of requiring a walled-garden distribution site that would separate us from direct contact with our users, and also being processor-specific. pNaCL eliminated the walled-garden requirement and added dynamic code compilation support, but still had the issues of being processor-specific code necessitating in our view device-specific testing, and a potentially long startup time as the code would be linked on first run. Finally, only working in Chrome would be a dealbreaker for our desire to have our game run in all major browsers.

A custom plugin/extension with C++ would require lots of testing & maintenance efforts on our part to run across different browsers, processor architectures, and operating systems, and such an installation requirement would likely scare away many potential players.

As it turned out, for our team’s purposes the Emscripten compiler & asm.js proved to be the best solution to these challenges, and when combined with a set of other new-ish Web API’s, enabled the browser to become a fully featured platform for instant high-end 3D gaming. This just took a little trial & error to figure out exactly how we’d piece it together…and that’s what we’ll be reviewing here!

First Steps into a Brave New World

We Trendy game engineers are primarily old-school C++ programmers, so it was something of a revelation that Emscripten could compile our existing application (built on Epic Game’s Unreal Engine 3) into asm.js optimized Javascript with little to no changes.

The primary Unreal Engine 3-specific code tweaks that were necessary to get the project to compile & run with Emscripten, were essentially… 1, 2, 3:

1.
// Esmcripten needs 4 byte alignment
FNameEntry* Allocate( INT Size )
{
    #if EMSCRIPTEN
       Size = Align( Size, 4 );
    #endif
 
    ......
}
 
2.
// Script execution: llvm needs aligned data
#if EMSCRIPTEN 
    #define XFER(T)
    {
        T Temp;
 
        if (!Ar.IsLoading())
        {
            appMemcpy(&Temp, &Script(iCode), sizeof(T));
        }
 
        Ar << Temp;
 
        if (!Ar.IsSaving())
        {
            appMemcpy(&Script(iCode), &Temp, sizeof(T));
        }
 
        iCode += sizeof(T);
    }
#else
    #define XFER(T) { Ar << *(T*)&Script(iCode); iCode += sizeof(T); }
#endif
 
3.
// This function needs to synchronously complete IO requests for single-threaded Emscripten IO to work, so added a ServiceRequestsSynchronously() to the end of it which flushes & blocks till the IO request finishes.
FAsyncIOSystemBase::QueueIORequest()

No really, that was about it! Within a day of fiddling with it, we had our game’s Javascript ‘executable’ compiled & running in the browser. Crashing, due to not having a graphics API implememented — but running with log output! Thankfully, we already had Unreal Engine3’s OpenGL ES2 version of the rendering subsystem ready to utilize, so porting the renderer to WebGL only took another day.

WebGL appeared to essentially have a superset of features compared to OpenGL ES2, so the shaders and methods used matched up by simply changing some API calls. In fact, we were able to do improvements by making use of WebGL’s floating point render targets for certain postprocessing effects, such as edge outlining and dynamic shadows.

EmscriptenGamePost
Postprocessing makes everything prettier!

But how’s it run?

Now we had something rendering in the browser, and with a quick change to capture input, we could start playing the game and analyzing its performance. What we found was very encouraging: straight ‘out of the box’, in Firefox the asm.js version of the game was getting nearly 33% of the performance of the native executable. And this was comparing a single-threaded web application to the multi-threaded native executable (so really, not a fair comparison! ;). This was about 2x the performance we saw with our quick Flash port (which we still utilize as a fallback for older browsers that don’t yet support asm.js, though we eventually hope to deprecate entirely).

Its performance in Chrome was less astonishing, more towards 20% of native performance, but still within our target margins: namely, can it run on a 2011-model Macbook Air at 45-60 FPS (with Vsync disabled)? The answer, thankfully, was yes. We hope Google will continue to improve the performance of asm.js on their browser over time. But as it currently stands, we believe unless you’re making the browser version of ‘Crysis’ with this tech (which may not be far off), it seems you have enough performance even in Chrome to do most kinds of web games.

BrowserFramerate
60 FPS on an old Macbook Air

Putting the Pieces into Place

So within a week from starting, we had turned our Unreal Engine 3 PC game into a well-running, graphically-rich web game. But where to take it from here? Well, it’d still need: Audio, Networking, Streaming, and Storage. Let’s discuss the various techniques used for each of these systems.

Audio

This was a no-brainer, as there is only really one robust standardized web audio system apart from Flash: WebAudio. Again, this API matched up pretty well to its mobile cousin, OpenSL, for which we already had an integration. So once we switched out the various calls, we had .

There was an apparent issue in Mac Chrome where sounds flagged “looping” would sometimes never become destroyed, so we implemented a Chrome-specific hack to manually loop the sound, and filed a bug report with Google. Ah well, one thing we’ve seen with browser API’s is there’s not a 100% guarantee that every browser will implement the functionality to perfect specification, but it gets the job done!

Networking

This proved a little trickier. First, we investigated WebRTC as used in the Bananabread demo, but WebRTC of course is for browser-to-browser communication which is actually not what we were looking to do. Our online game service uses a server-and-client architecture with centralized infrastructure, and so WebSockets is the API to utilize in that case. The tricky part is that we have to handle all the WebSockets incoming and outgoing data in JavaScript buffers, and then pass that along to the “C++” Emscripten-compiled game.

With some callbacks, this worked out, but we also had to take our UDP game server code and place the WebSockets TCP-style layer onto it — some iteration was necessary to get the packets to be formatted in exactly the way that WebSockets expects, but once we did that, our browser game was communicating with our backend-hosted Linux dedicated game servers with no problems!

Streaming & Storage

One advantage to being on the web is easy access to the browser’s asynchronous downloading functionalities to stream-in content. We certainly made use of this with our game, with the initial download clocking in at under 10 MB. Everything else streams in on-demand as you play using standard browser http download requests: Textures, Sound Effects, Music, even Skeletal Meshes and Level Packages. But the bigger question is how to reliably store this content. We don’t want to just rely on the browser cache, since it’s not good for guaranteed immediate gameplay loading as we can’t pre-query whether something exists on disk in the regular browser cache or not.

For this, we used the IndexedDB API, which lets us asynchronously save and retrieve data objects from a secure abstracted storage location. It works in both Chrome and Firefox, though it’s still finicky as the database can occasionally become corrupted (perhaps if terminated during async writes) and has to be regenerated. In the worst case, this simply results in a re-download of content the user already had received.

We’re currently looking into this issue, but that aside, IndexedDB certainly works well and has the advantage of providing our application standard file IO functionality, useful to store content that we download. (UPDATE: Firefox Nightly build as of 12/10 seems to automatically reset the IndexedDB storage if this happens and it may not recur.)

Play it Now, and Embrace the Future!

While we still have more profiling and tweaking to, as we’re just now starting to use Firefox’s VTune support to symbolically profile the asm.js performance within the browser. Even still, we’re pretty pleased with where the things currently stand. But don’t take our word for it, please try it yourselves right here, no installation or sign-up required:

Try our demo test anonymously In-browser Here!
(Please bear with us if our game servers limit access under load, we’re still testing our backend scalability!)

We at Trendy envision a day when anybody can play any game no matter where they are or what device they happen to have, without friction or gateways or middlemen. With the right combination of these cutting-edge web technologies, that day can be today. We hope other enterprising game developers will join us in reaching players directly through the web, which thanks to Emscripten & asm.js, may well become the most powerful and far-reaching “game console” of all!

11 comments

Comments are now closed.

  1. Tom wrote on December 12th, 2013 at 11:28:

    Very interesting indeed!!! Still, a part the learning side of the thing, I wonder, what about using Unity3D which has a cross platform fast native plug-in AND the support to Google Chrome native client?

    1. Jeremy Stieglitz wrote on December 14th, 2013 at 21:45:

      @ Tom,

      The main benefits of Emscripten vs the Unity Plugin+NaCl are two-fold in my view:

      - On browsers where you’d have to go with the Unity plugin, having the plugin install requirement may lose you a lot of potential players. From some folks who work at Trendy Entertainment who previously created Unity web games, they tell me their user attrition rate at the plugin gate was as high as 75%. Even though Unity’s a fairly “well known” plugin, to the average user it is still a potentially scary proposition installing something they may have never heard of before.

      - By using Emscripten, you don’t have to be locked into just using Unity. As nice as it is, there are other powerful game engines out there too which provide full source code to compile in Emscripten, or maybe you’d like to create your own to make a nifty application :)

      Also, Emscripten use doesn’t have to just be for entire web applications, it could also be theoretically used to make web-hosted functional libraries that do heavy performance-sensitive internal work with inut, and then feed output data back to any web application or Unity plugin game via JavaScript. We soon intend to use this approach to standardize our networking library, for example.

      I will say, in my own personal view Unity would benefit from having an Emscripten web version as an deployment option. Hopefully someone over at Unity HQ may be reading this!

      -Jeremy

  2. Yuri wrote on December 13th, 2013 at 00:09:

    Great tech stack! However, I couldn’t login to the game due to blocking of non-standard ports by our corporate proxy. Why not just using 80/443 ports for networking?

    1. Jeremy Stieglitz wrote on December 14th, 2013 at 21:38:

      @ Yuri,

      Actually, there is no reason we don’t run the WebSockets on a standard port like port 80! That was our oversight, and we’re going to fix it on Monday! I’ll post here / update the blog when it’s moved to port 80 and then you should be able to try the game behind any corporate firewall, mwahaha!

      -Jeremy

  3. Christian wrote on December 13th, 2013 at 05:00:

    @Tom:
    Compiling the code for each architecture and lots of native code implementations or using unitiy3d, which is great for easy beginning, is not nearly that impressive as bringing the whole game to platform independent javascript with this performance!
    Keep in mind that this solution is using open standards and no proprietary base like unitiy3d, no license fees at all and accessible for everyone.

    1. Tom wrote on January 2nd, 2014 at 17:17:

      I get your point, and I agree with you.
      I was astonished by the great hack, and if it’s fast! Ok!
      But isn’t that still Unreal Technology ?

  4. Andre Jaenisch wrote on December 13th, 2013 at 07:58:

    Hello,

    I’ve pointed Bob Thulfram from firefoxosgaming.blogspot.com to this article.
    But currently (15:57 GMT+1:00) there seems to be username+password required, so I couldn’t test it.

    Debian GNU/Linux with Firefox 26.0

  5. Paul Irish wrote on December 13th, 2013 at 12:18:

    Great writeup!

    A note on pNaCL. Above, it says “only working in Chrome would be a dealbreaker for our desire to have our game run in all major browsers.”

    Luckily, that’s no longer true. Pepper.js, which is fairly new, will transpile any pNaCL application into JS through Emscripten to run on all major browsers: http://trypepperjs.appspot.com/

    Also curious, what’s blocking you from putting Monster Madness on the mobile web? I think it’s fair to say that both Firefox and Chrome (which support WebGL & Web Audio) want to enable you to do that. :)

    1. Jeremy Stieglitz wrote on December 14th, 2013 at 21:37:

      @ Paul,

      Thank you for the heads-up on pepper.js! That’s a very interesting approach and we’re going to try it out here. I’d still have potential test-oriented concerns about the need to try out the pNaCl compiled application on the various CPU architectures to ensure there aren’t any linktime/runtime issues, but perhaps I’m overestimating the possibility of that to cause issues. Definitely the potential of pNaCl applications to run in other browsers via Emscripten makes it much more attractive… Emscripten saving the day again? ;)

      -Jeremy

  6. v792933579 wrote on December 19th, 2013 at 04:31:

    browser ran out of memory,please restart

  7. Karsten Bruch wrote on December 29th, 2013 at 07:13:

    Was this
    http://www.playverse.com/Anonplayer/0-a2aadd1b76e14d0e848ea1de18dca4e8
    temporary ?

    Says right now:
    “Anonymous accounts are not enabled for this game!”

    Cheers,
    Karsten

Comments are closed for this article.