This post is by Felipe Gomez, a long-time Mozilla contributor and really awesome dude. Felipe attends school in Brazil.
The effect is quite simple. What we want to do is to create a stamp effect based on the brightness of the image. We calculate the brightness of each pixel and separate them into 3 groups, from darkest to brightest. Them, for each of our four frames we use a different color palette to convert the original pixels into our desired colors for the best effect.
With that, we already have a cool effect, but it lacks a bit of detail:
So we mix our colors with 30% of the original image, and now we have the final version, with more shades of color while preserving the nice colorization in Andy Warhol style.
We can access the pixels of each frame of the video using <video> and <canvas>, creating a live effect while the video plays.