This post is by Vladimir Vukićević and is a re-post from his personal weblog.
I get about 15-16 frames per second with the default zoomed out image (around 5 million pixels per second — that number won’t be affected by image size) on my MacBook Pro, which is definitely fast enough for live manipulation. The algorithm could be tightened up to make this faster still. Further optimizations to the JS engine could help here as well; for example, I noticed that we spend a lot of time doing floating point to integer conversions for writing the computed pixels back to the display canvas, due to how the canvas API specifies image data handling.
The Web Darkroom tool also supports drag & drop, so you can take any image from your computer and drop it onto the canvas to load it. A long (long!) time ago, back in 2006, I wrote an addon called “Croppr!”. It was intended to be used with Flickr, allowing users to play around with custom crops of any image, and then leave crop suggestions in comments to be viewed using Croppr. It almost certainly doesn’t work any more, but it would be neat to update it: this time with both cropping and color correction. Someone with the addon (perhaps a Jetpack now!) could then visit a Flickr photo and experiment, and leave suggestions for the photographer.
The second example is based on some work that Dave Humphrey and others have been doing to bring audio manipulation to the web platform. Originally, their spec included a pre-computed FFT with each audio frame delivered to the web app. I suggested that there’s no need for this — while a FFT is useful for some applications, for others it would be wasted work. Those apps that want a FFT could implement one in JS. Some benchmark numbers backed this up — using the typed arrays originally created for WebGL, computing an FFT in JS was approaching the speed of native code. Again, both could be sped up (perhaps using SSE2 or something like Mono.Simd on the JS side), but it’s fast enough to be useful already.
The demo shows this in action. A numeric benchmark isn’t really all that interesting, so instead I take a video clip, and as it’s playing, I extract a portion of the green channel of each frame and compute its 2D FFT, which is then displayed. The original clip plays at 24 frames per second, so that’s the upper bound of this demo. Using Float32 typed arrays, the computation and playback proceeds at around 22-24fps for me.
You can grab the video controls and scrub to a specific frame. (The frame rate calculation is only correct while the video is playing normally, not while you’re scrubbing.) The video source uses Theora, so you’ll need a browser that can play Theora content. (I didn’t have a similar clip that uses WebM, or I could have used that.)
Edit: Made some last-minute changes to the demos, which ended up pulling in a slightly broken version of jQuery UI that wasn’t all that happy with Safari. Should be fixed now!