ES6 In Depth: Proxies

ES6 In Depth is a series on new features being added to the JavaScript programming language in the 6th Edition of the ECMAScript standard, ES6 for short.

Here is the sort of thing we are going to do today.

var obj = new Proxy({}, {
  get: function (target, key, receiver) {
    console.log(`getting ${key}!`);
    return Reflect.get(target, key, receiver);
  },
  set: function (target, key, value, receiver) {
    console.log(`setting ${key}!`);
    return Reflect.set(target, key, value, receiver);
  }
});

That’s a little complicated for a first example. I’ll explain all the parts later. For now, check out the object we created:

> obj.count = 1;
    setting count!
> ++obj.count;
    getting count!
    setting count!
    2

What’s going on here? We are intercepting property accesses on this object. We are overloading the "." operator.

How it’s done

The best trick in computing is called virtualization. It’s a very general-purpose technique for doing astonishing things. Here’s how it works.

  1. Take any picture.

    (picture of a coal power plant)

    Photo credit: Martin Nikolaj Bech
  2. Draw an outline around something in the picture.

    (same photo, with the power plant circled)
  3. Now replace either everything inside the outline, or everything outside the outline, with something totally unexpected. There is just one rule, the Rule of Backwards Compatibility. Your replacement must behave enough like what was there before that nobody on the other side of the line notices that anything has changed.

    (the circled part is replaced with a wind farm)

    Photo credit: Beverley Goodwin.

You’ll be familiar with this kind of hack from classic computer science films such as The Truman Show and The Matrix, where a person is inside the outline, and the rest of the world has been replaced with an elaborate illusion of normalcy.

In order to satisfy the Rule of Backwards Compatibility, your replacement may need to be cunningly designed. But the real trick is in drawing the right outline.

By outline, I mean an API boundary. An interface. Interfaces specify how two bits of code interact and what each part expects of the other. So if an interface is designed into the system, the outline is already drawn for you. You know you can replace either side, and the other side won’t care.

It’s when there’s not an existing interface that you have to get creative. Some of the coolest software hacks of all time have involved drawing an API boundary where previously there was none, and bringing that interface into existence via a prodigious engineering effort.

Virtual memory, Hardware virtualization, Docker, Valgrind, rr—to various degrees all of these projects involved driving new and rather unexpected interfaces into existing systems. In some cases, it took years and new operating system features and even new hardware to make the new boundary work well.

The best virtualization hacks bring with them a new understanding of whatever’s being virtualized. To write an API for something, you have to understand it. Once you understand, you can do amazing things.

ES6 introduces virtualization support for JavaScript’s most fundamental concept: the object.

What is an object?

No, really. Take a moment. Think it over. Scroll down when you know what an object is.

(picture of Auguste Rodin’s sculpture, The Thinker)

Photo credit: Joe deSousa.

This question is too hard for me! I’ve never heard a really satisfying definition.

Is that surprising? Defining fundamental concepts is always hard—check out the first few definitions in Euclid’s Elements sometime. The ECMAScript language specification is in good company, therefore, when it unhelpfully defines an object as a “member of the type Object.”

Later, the spec adds that “An Object is a collection of properties.” That’s not bad. If you want a definition, that will do for now. We’ll come back to it later.

I said before that to write an API for something, you have to understand it. So in a way, I’ve promised that if we get through all this, we’re going to understand objects better, and we’ll be able to do amazing things.

So let’s follow in the footsteps of the ECMAScript standard committee and see what it would take to define an API, an interface, for JavaScript objects. What sort of methods do we need? What can objects do?

That depends somewhat on the object. DOM Element objects can do certain things; AudioNode objects do other things. But there are a few fundamental abilities all objects share:

  • Objects have properties. You can get and set properties, delete them, and so on.
  • Objects have prototypes. This is how inheritance works in JS.
  • Some objects are functions or constructors. You can call them.

Almost everything JS programs do with objects is done using properties, prototypes, and functions. Even the special behavior of an Element or AudioNode object is accessed by calling methods, which are just inherited function properties.

So when the ECMAScript standard committee defined a set of 14 internal methods, the common interface for all objects, it should come as no surprise that they ended up focusing on these three fundamental things.

The full list can be found in tables 5 and 6 of the ES6 standard. Here I’ll just describe a few. The weird double brackets, [[ ]], emphasize that these are internal methods, hidden from ordinary JS code. You can’t call, delete, or overwrite these like ordinary methods.

  • obj.[[Get]](key, receiver) – Get the value of a property.

    Called when JS code does: obj.prop or obj[key].

    obj is the object currently being searched; receiver is the object where we first started searching for this property. Sometimes we have to search several objects. obj might be an object on receiver’s prototype chain.

  • obj.[[Set]](key, value, receiver) – Assign to a property of an object.

    Called when JS code does: obj.prop = value or obj[key] = value.

    In an assignment like obj.prop += 2, the [[Get]] method is called first, and the [[Set]] method afterwards. Same goes for ++ and --.

  • obj.[[HasProperty]](key) – Test whether a property exists.

    Called when JS code does: key in obj.

  • obj.[[Enumerate]]() – List obj’s enumerable properties.

    Called when JS code does: for (key in obj) ....

    This returns an iterator object, and that’s how a forin loop gets an object’s property names.

  • obj.[[GetPrototypeOf]]() – Return obj’s prototype.

    Called when JS code does: obj.__proto__ or Object.getPrototypeOf(obj).

  • functionObj.[[Call]](thisValue, arguments) – Call a function.

    Called when JS code does: functionObj() or x.method().

    Optional. Not every object is a function.

  • constructorObj.[[Construct]](arguments, newTarget) – Invoke a constructor.

    Called when JS code does: new Date(2890, 6, 2), for example.

    Optional. Not every object is a constructor.

    The newTarget argument plays a role in subclassing. We’ll cover it in a future post.

Maybe you can guess at some of the other seven.

Throughout the ES6 standard, wherever possible, any bit of syntax or builtin function that does anything with objects is specified in terms of the 14 internal methods. ES6 drew a clear boundary around the brains of an object. What proxies let you do is replace the standard kind of brains with arbitrary JS code.

When we start talking about overriding these internal methods in a moment, remember, we’re talking about overriding the behavior of core syntax like obj.prop, builtin functions like Object.keys(), and more.

Proxy

ES6 defines a new global constructor, Proxy. It takes two arguments: a target object and a handler object. So a simple example would look like this:

var target = {}, handler = {};
var proxy = new Proxy(target, handler);

Let’s set aside the handler object for a moment and focus on how proxy and target are related.

I can tell you how proxy is going to behave in one sentence. All of proxy’s internal methods are forwarded to target. That is, if something calls proxy.[[Enumerate]](), it’ll just return target.[[Enumerate]]().

Let’s try it out. We’ll do something that causes proxy.[[Set]]() to be called.

proxy.color = "pink";

OK, what just happened? proxy.[[Set]]() should have called target.[[Set]](), so that should have made a new property on target. Did it?

> target.color
    "pink"

It did. And the same goes for all the other internal methods. This proxy will, for the most part, behave exactly the same as its target.

There are limits to the fidelity of the illusion. You’ll find that proxy !== target. And a proxy will sometimes flunk type checks that the target would pass. Even if a proxy’s target is a DOM Element, for example, the proxy isn’t really an Element; so something like document.body.appendChild(proxy) will fail with a TypeError.

Proxy handlers

Now let’s return to the handler object. This is what makes proxies useful.

The handler object’s methods can override any of the proxy’s internal methods.

For example, if you’d like to intercept all attempts to assign to an object’s properties, you can do that by defining a handler.set() method:

var target = {};
var handler = {
  set: function (target, key, value, receiver) {
    throw new Error("Please don't set properties on this object.");
  }
};
var proxy = new Proxy(target, handler);

> proxy.name = "angelina";
    Error: Please don't set properties on this object.

The full list of handler methods is documented on the MDN page for Proxy. There are 14 methods, and they line up with the 14 internal methods defined in ES6.

All handler methods are optional. If an internal method is not intercepted by the handler, then it’s forwarded to the target, as we saw before.

Example: “Impossible” auto-populating objects

We now know enough about proxies to try using them for something really weird, something that’s impossible without proxies.

Here’s our first exercise. Make a function Tree() that can do this:

> var tree = Tree();
> tree
    { }
> tree.branch1.branch2.twig = "green";
> tree
    { branch1: { branch2: { twig: "green" } } }
> tree.branch1.branch3.twig = "yellow";
    { branch1: { branch2: { twig: "green" },
                 branch3: { twig: "yellow" }}}

Note how all the intermediate objects branch1, branch2, and branch3, are magically autocreated when they’re needed. Convenient, right? How could it possibly work?

Until now, there’s no way it could work. But with proxies this is only a few lines of code. We just need to tap into tree.[[Get]](). If you like a challenge, you might want to try implementing this yourself before reading on.

(picture of a tap in a maple tree)

Not the right way to tap into a tree in JS. Photo credit: Chiot’s Run.

Here’s my solution:

function Tree() {
  return new Proxy({}, handler);
}

var handler = {
  get: function (target, key, receiver) {
    if (!(key in target)) {
      target[key] = Tree();  // auto-create a sub-Tree
    }
    return Reflect.get(target, key, receiver);
  }
};

Note the call to Reflect.get() at the end. It turns out there’s an extremely common need, in proxy handler methods, to be able to say “now just do the default behavior of delegating to target.” So ES6 defines a new Reflect object with 14 methods on it that you can use to do exactly that.

Example: A read-only view

I think I may have given the false impression that proxies are easy to use. Let’s do one more example to see if that’s true.

This time our assignment is more complex: we have to implement a function, readOnlyView(object), that takes any object and returns a proxy that behaves just like that object, except without the ability to mutate it. So, for example, it should behave like this:

> var newMath = readOnlyView(Math);
> newMath.min(54, 40);
    40
> newMath.max = Math.min;
    Error: can't modify read-only view
> delete newMath.sin;
    Error: can't modify read-only view

How can we implement this?

The first step is to intercept all internal methods that would modify the target object if we let them through. There are five of those.

function NOPE() {
  throw new Error("can't modify read-only view");
}

var handler = {
  // Override all five mutating methods.
  set: NOPE,
  defineProperty: NOPE,
  deleteProperty: NOPE,
  preventExtensions: NOPE,
  setPrototypeOf: NOPE
};

function readOnlyView(target) {
  return new Proxy(target, handler);
}

This works. It prevents assignment, property definition, and so on via the read-only view.

Are there any loopholes in this scheme?

The biggest problem is that the [[Get]] method, and others, may still return mutable objects. So even if some object x is a read-only view, x.prop may be mutable! That’s a huge hole.

To plug it, we must add a handler.get() method:

var handler = {
  ...

  // Wrap other results in read-only views.
  get: function (target, key, receiver) {
    // Start by just doing the default behavior.
    var result = Reflect.get(target, key, receiver);

    // Make sure not to return a mutable object!
    if (Object(result) === result) {
      // result is an object.
      return readOnlyView(result);
    }
    // result is a primitive, so already immutable.
    return result;
  },

  ...
};

This is not sufficient either. Similar code is needed for other methods, including getPrototypeOf and getOwnPropertyDescriptor.

Then there are further problems. When a getter or method is called via this kind of proxy, the this value passed to the getter or method will typically be the proxy itself. But as we saw earlier, many accessors and methods perform a type check that the proxy won’t pass. It would be better to substitute the target object for the proxy here. Can you figure out how to do it?

The lesson to draw from this is that creating a proxy is easy, but creating a proxy with intuitive behavior is quite hard.

Odds and ends

  • What are proxies really good for?

    They’re certainly useful whenever you want to observe or log accesses to an object. They’ll be handy for debugging. Testing frameworks could use them to create mock objects.

    Proxies are useful if you need behavior that’s just slightly past what an ordinary object can do: lazily populating properties, for example.

    I almost hate to bring this up, but one of the best ways to see what’s going on in code that uses proxies… is to wrap a proxy’s handler object in another proxy that logs to the console every time a handler method is accessed.

    Proxies can be used to restrict access to an object, as we did with readOnlyView. That sort of use case is rare in application code, but Firefox uses proxies internally to implement security boundaries between different domains. They’re a key part of our security model.

  • Proxies ♥ WeakMaps. In our readOnlyView example, we create a new proxy every time an object is accessed. It could save a lot of memory to cache every proxy we create in a WeakMap, so that however many times an object is passed to readOnlyView, only a single proxy is created for it.

    This is one of the motivating use cases for WeakMap.

  • Revocable proxies. ES6 also defines another function, Proxy.revocable(target, handler), that creates a proxy, just like new Proxy(target, handler), except this proxy can be revoked later. (Proxy.revocable returns an object with a .proxy property and a .revoke method.) Once a proxy is revoked, it simply doesn’t work anymore; all its internal methods throw.

  • Object invariants. In certain situations, ES6 requires proxy handler methods to report results that are consistent with the target object’s state. It does this in order to enforce rules about immutability across all objects, even proxies. For example, a proxy can’t claim to be inextensible unless its target really is inextensible.

    The exact rules are too complex to go into here, but if you ever see an error message like "proxy can't report a non-existent property as non-configurable", this is the cause. The most likely remedy is to change what the proxy is reporting about itself. Another possibility is to mutate the target on the fly to reflect whatever the proxy is reporting.

What is an object now?

I think where we left it was: “An Object is a collection of properties.”

I’m not totally happy with this definition, even taking for granted that we throw in prototypes and callability as well. I think the word “collection” is too generous, given how poorly defined a proxy can be. Its handler methods could do anything. They could return random results.

By figuring out what an object can do, standardizing those methods, and adding virtualization as a first-class feature that everyone can use, the ECMAScript standard committee has expanded the realm of possibilities.

Objects can be almost anything now.

Maybe the most honest answer to the question “What is an object?” now is to take the 12 required internal methods as a definition. An object is something in a JS program that has a [[Get]] operation, a [[Set]] operation, and so on.


Do we understand objects better after all that? I’m not sure! Did we do amazing things? Yeah. We did things that were never possible in JS before.

Can I use Proxies today?

Nope! Not on the Web, anyway. Only Firefox and Microsoft Edge support proxies, and there is no polyfill.

Using proxies in Node.js or io.js requires both an off-by-default option (--harmony_proxies) and the harmony-reflect polyfill, since V8 implements an older version of the Proxy specification. (A previous version of this article had incorrect information about this. Thanks to Mörre and Aaron Powell for correcting my mistakes in the comments.)

So feel free to experiment with proxies! Create a hall of mirrors where there seem to be thousands of copies of every object, all alike, and it’s impossible to debug anything! Now is the time. There’s little danger of your ill-advised proxy code escaping into production… yet.

Proxies were first implemented in 2010, by Andreas Gal, with code reviews by Blake Kaplan. The standard committee then completely redesigned the feature. Eddy Bruel implemented the new spec in 2012.

I implemented Reflect, with code reviews by Jeff Walden. It’ll be in Firefox Nightly starting this weekend—all except Reflect.enumerate(), which is not implemented yet.

Next up, we’ll be talking about the most controversial feature in ES6, and who better to present it than the person who’s implementing it in Firefox? So please join us next week as Mozilla engineer Eric Faust presents ES6 classes in depth.

About Jason Orendorff

More articles by Jason Orendorff…


13 comments

  1. simonleung

    The usage of Proxy is similar to Object.defineProperty but it is more comprehensive and powerful. It nearly let you define all behaviors of an object.

    Here are some syntax you will never see before:
    document.body.css.color(“red”).backgroundColor=”blue”;
    document.body.appendChild(document.createElement(“div”).css.backgroundColor(“red”).border(“5px solid green”).width(“500px”).height(“500px”)());
    Note: css is a property of HTMLElement.prototype defined by Object.defineProperty, which return a proxy of element’s style object.

    Instead of getAttribute, setAttribute, removeAttriute in DOM, a proxy can let you do the same like this:
    element.attr.newAttr=”123”;
    alert(element.attr.newAttr);
    delete element.attr.newAttr;

    Similarly, document.body.on.click(function(e){alert(“clicked”)},true);

    Note: on and attr are proxy.

    A canvas object proxy:
    .moveTo(100,10).lineTo(130,10)(130,40)(100,40).closePath().stroke();

    July 18th, 2015 at 07:10

  2. Mörre

    > Only Firefox supports proxies, and there is no polyfill.

    This has just recently been released:

    https://github.com/tvcutsem/harmony-reflect

    On node.js/io.js it builds on the pre-existing proxy support enabled with –harmony_proxies

    July 19th, 2015 at 03:37

    1. Jason Orendorff

      Oh, interesting! I didn’t know this was out there, obviously. I’ll update the article.

      July 29th, 2015 at 16:05

  3. Aaron Powell

    MS Edge also supports proxies completely.

    July 19th, 2015 at 20:01

    1. Jason Orendorff

      Oops. I’ll update the article!

      July 29th, 2015 at 12:10

  4. bystander

    @simonleung true. I have to say I have mixed feelings about all this, having experienced the PHP magic functions (if you haven’t done PHP, this is literally how they are called in the documentation). They are similar to proxies and enable interception of non-existent properties and methods, etc.

    On one hand you can create cool dynamic objects that really behave like magic if you know the right micro-language (because this is what it really is). E.g. in PHP the data layer allowed to do: client.findById, client.findByName, where of course none of those methods actually existed. You could findBy anything!

    On the other hand it is extremely frustrating at design time. You usually get poor or no completion when typing, no static verification when compiling (if you compile…) and when things break down you just have no idea what happens, it’s a pain to try to debug.

    So… mixed feelings as I said.

    BTW, most of your examples could be done today statically, if one defined the right objects with the right methods. Your “.on” or “.css” proxies could exist today with static methods (lots of them, though). Actually, the “.style” property is kind of your “.css” proxy, with properties rather than methods — not fluently chainable :( But it shows that it’s doable, with design-time support, etc. That got me thinking.

    July 20th, 2015 at 14:43

  5. Boris Prpić

    So how proxies relate to Object.observe() ?
    Can they be used by databinding frameworks for dirty checking?

    July 20th, 2015 at 17:12

  6. karthick siva

    Thanks for the detailed post jason. I am confusing myself between proxy handler and reflect.

    “now just do the default behavior of delegating to target.” If reflect doing the default behavior then why do we need it? Is it possible to have a proxy handler without reflect?

    July 27th, 2015 at 02:02

    1. Jason Orendorff

      Let’s take Reflect.get() as an example.

      If a proxy handler has no handler.get(target, id, receiver) method, then the default behavior is exactly what Reflect.get(target, id, receiver) does. In that case, you don’t need Reflect.

      If you don’t want the default behavior at all, that’s easy enough: implement a handler.get() method. The default behavior will not happen. You still don’t need Reflect.

      Reflect is useful when you want to have a special handler.get() method, and you *also* want it to perform the default behavior on the target, or you want to do something special conditionally, falling back on default behavior in all other cases.

      July 29th, 2015 at 13:22

      1. karthick siva

        That clarifies my doubt. thanks…

        July 29th, 2015 at 22:43

  7. Luke

    “Until now, there’s no way it could work. But with proxies this is only a few lines of code. ” – really?? “If you like a challenge, you might want to try implementing this yourself before reading on.”

    Challenge accepted… and it uses currently-supported JS:
    http://jsfiddle.net/cjsa92w0/1/

    The correct values are logged to console. Repeat for any other properties that are expected to be in the object.
    Similarly, you can also already make a read-only property with defineProperty().

    “They’ll be handy for debugging. Test frameworks could use them to mock real objects.”
    Ok, but why not use Object.watch() to see when the actual objects change? Other than the fact that Object.watch is Mozilla-only, and I assume these Proxies will be standardized soon.

    These are interesting features coming but not actually that different from what we have. I assume these ES6 native settings may be faster than their current equivalents in Backbone and current JS?

    July 28th, 2015 at 20:01

    1. Jason Orendorff

      Unlike Object.defineProperty(), Proxies can override behavior for all properties of an object. This is why the line of code `tree.branch1.branch3.twig = “yellow”;` from the article works for the Proxy-based Tree, but doesn’t work in your codepen.

      Of course we could keep adding more code, to pre-define “branch3” and every other possible property our users may need. But at some point that gets a bit silly, right?

      Unlike Object.prototype.watch(), Proxies can override all property accesses, not just assignment. Also, Proxies are standard now, since ECMA has ratified ES6.

      To me, they seem like a pretty radical departure from anything that exists in Backbone or current JS.

      July 29th, 2015 at 12:08

  8. simonleung

    What is an object now? It is not a good question.
    You should ask “What does the dot mean now?” It is defined by yourself by Proxy.

    http://jsfiddle.net/oa5scs4r/

    It is impossible in ES5.

    July 29th, 2015 at 10:45

Comments are closed for this article.