Mozilla

People of HTML5 Articles

Sort by:

View:

  1. People of HTML5: Andrew Betts on building the FT.com HTML5 app

    HTML5 needs spokespeople to work. There are a lot of people out there who took on this role, and here at Mozilla we thought it is a good idea to introduce some of them to you with a series of interviews and short videos. The format is simple – we send the experts 10 questions to answer and then do a quick video interview to let them introduce themselves and ask for more detail on some of their answers.

    Andrew Betts Today we are featuring Andrew Betts, director of Assanka.

    We came across Andrew because of his talk “We’ve Got a Website for That – The FT Web App and the Future of the Mobile Web” at the BlackBerry Devcon in Amsterdam where he did a great job allowing us to peek under the hood of the Financial Times web app. You can reach Andrew on Twitter as @triblondon.

    The video interview

    You can see the video on any HTML5 enabled device here (courtesy of vid.ly).

    10 Questions for Andrew Betts

    1) You worked on the HTML5 application for the financial times. What made you go HTML5 and not native? How many people were involved at all?

    The FT has an agnostic approach when it comes to the formats and technologies we can use to display our content. The choice has to suit the content, the reader, and our business model. Before we launched the HTML5 app there was already an FT native app on iOS, and it received an Apple design award, but HTML5 offered several significant benefits, the most important of which was to allow us to maintain a direct relationship with the reader. Secondary benefits include a faster update process, an investment in future cross-platform compatibility (HTML5 is yet to fully realise the dream of write-once, run everywhere), and freedom from restrictions and rules that are (or may be in future) imposed by app store operators.

    2) What would you say was the easiest part and what was the thing that you had to spend the most time on to make it work?

    Interactions are the hardest thing to crack. Matching behaviour to user expectations is frighteningly complex, especially when you realise that instinctive expectations of swipe behaviour is actually quite varied and inconsistent. One example of this complexity is swiping horizontally from the bottom of a section page, which takes you to the top of the next one. If you then swipe back, do you end up where you were, or at the top? So dealing with interactions is both technically hard and architecturally challenging.

    Offline behaviour is also very tough. We get a lot of “User x can’t use the app offline” style support requests, and those are very hard to debug. We log every report but the priority is to figure out the scale of the problem.

    The easiest thing to deal with is probably layout. We send all legacy browsers to m.ft.com so we only have to cope with the most recent major versions, and that means that generally we get good standards support for CSS.

    3) How did you deal with the differences in interfaces? A mobile has much less space than a tablet and again less than a desktop. Are you using mediaqueries to give different experiences?

    We group devices into one of four sizes based on media queries, and we vary which resources we use slightly based on the reported DPI, but most of our responsive layout logic is done in JavaScript. We make a few assumptions right now to make development easier, such as that the user cannot change the size of the browser window, except by rotating the device, so we only need to change the layout on orientation change, not in response to arbitrary resizing of the window. We also currently assume the user has no mouse cursor, so there are no hover effects.

    4) How does input get into it? Is it simple enough to build touch interfaces in HTML5 or did you have to use some trickery?

    Quite a lot of trickery is involved, unfortunately. The browser has to tread a fine line between on the one hand offering a touch API to web developers and getting out of their way to let them use it, and on the other, dealing with the fact that almost all websites are still built for keyboard and mouse, so the browser needs to help the user deal with that. We developed a polyfill called Fastclick which makes click events fire as fast as touch events, and that certainly helps us provide an experience that is as snappy as using a native app.

    We also deal with swiping in a few quite distinctly different ways. Swiping between sections is an endless carousel, and we implement the tracking and sliding ourselves, by maintaining hidden page containers to the left and right of the current view and applying CSS 3D transforms. But paging through an article requires different scroll mechanics – it has a fixed length, and we have to lay out the whole article in one go to know how many pages there are. Pages that scroll vertically are a different challenge again, and there we use a combination of leaving the browser to do scrolling natively, and simulating it where we need slightly different behaviour.

    You’ll also notice that the pages don’t ‘bounce’ when you hit the top and bottom of the page, despite that being the normal behaviour in the iOS browser. We’ve gone to quite some lengths to ensure that the user experience mirrors an app and doesn’t feel like the device ‘coping’ with a site that hasn’t been designed to be viewed using a device with touch input.

    5) You said in your talk at Blackberry Devcon that you are using localStorage to cache your JavaScript libraries and some of the images. Why not use WebSQL or IndexDB for that? Isn’t the synchronous nature of localStorage slowing things down?

    No, actually localStorage is considerably faster than WebSQL (we are yet to use IndexedDB because it’s not supported on some of our target platforms). The time required for a lookup by key in localStorage is many times less than the equivalent SELECT statement on a WebSQL table. It’s true that it’s synchronous, but if the critical aspect affecting perceived performance is the time required to fetch data, which it often is, then localStorage offers a much faster response. it’s also typically reliable and relatively stable compared to the behaviour of WebSQL and particularly the HTML5 app cache.

    6) You also said that localStorage uses UTF-16 instead of UTF-8. This means you can store two ASCII characters in one UTF-16 one and thus double the storage capacity, right? Can you share some code? This would be incredible to use for everybody out there.

    Both localStorage and WebSQL use UTF-16 in webkit browsers, it’s one of many aspects of the implementation of these technologies that don’t make a lot of sense! We’re experimenting with some algorithms for compressing our data so we can make more efficient use of that storage, and one of those is to combine two characters into a single UTF-16 one. We’re not using that in production yet, but when we do we’re certainly hoping to open source it.

    There are simpler things to be done as well. A cursory poke and prod of WebSQL tables suggest that column names are stored along with the value in every cell, so some attempts to increase efficiency are as simple as using single character column names. Clearly we ought not to have to do all this, but it’s the price you pay for dealing with bleeding edge features in browsers.

    7) What would you consider the best start for someone when they want to build an HTML5 app? What are the first things to get sorted?

    Decide ahead of time what you’re trying to achieve. I could be a pedant and say that making your site HTML5 is really just a matter of changing the DOCTYPE, so an ‘HTML5 app’ is really whatever you define it to be, and having a clear idea of that is a good place to begin. Are you building something just for touch, or for keyboard and mouse as well? Does it need to work offline? Will it be crawlable by search spiders? Will it work with JavaScript disabled?

    There are some great resources like MDN, but we often find ourselves reading the HTML5 specs, and sometimes even the webkit source code to find out how something is actually implemented.

    8) Can native and web apps be the same thing?

    Technically, I guess not – strictly speaking a native app is complied code running directly on a platform, but the hybrid model seems to be getting ever more popular, and I often see apps which are thinly veiled web browsers. If you look at it from the user perspective and say, can a web app feel exactly like a native app, I don’t see why not. The challenges are greater because of the diversity of uses that web technologies are designed for, and the diversity of platforms on which they are used. And for uses such as gaming, native code is always likely to run faster and have deeper access to the OS.

    But for applications like publishing, which after all was the original purpose of the web, web technologies do provide support for most of the user experience concepts that we need.

    9) What about testing? How did you approach this? Do you have an array of hardware to play with?

    It’s a nightmare. We had a local electrician built us a charging station where we could store all our devices and keep them charged all the time. We have around 45 individual devices in our team (that’s around 4 times more devices than we have people). We’re constantly looking at ways of improving our testing process, and we keep revisiting automated, on device testing, but we’re not using it in our build cycle as yet. Right now we have a team of very dedicated testers who poke and prod devices all day.

    10) As someone who went through the process of building a big HTML5 app, what would you like to have from browser vendors to make your life easier?

    A blithe answer is ‘How long do you have’, but in practice we have to accept we’re using freshly minted technologies and there might be teething troubles. The main things on our shopping list are: more frequent and aggressive browser updates (especially on Android, where the browser is the problem child of the mobile web world), a better and more reliable app cache, and hardware acceleration of CSS transforms.

  2. Moving browsers and the web forward (video)

    A few days ago I was asked to deliver the first talk of the amazing Beyond Tellerand conference in Dusseldorf, Germany. The talk Breaking the barriers – moving browsers and the web forward introduced a lot of new ideas and technologies that are worked on my Mozilla and others to make the web of the future better.

    Here is the video of the presentation with jump points and links to more information. If you want to see the slides of the talk, they are available here and there is also an audio recording on archive.org.

    Breaking the barriers – moving browsers and the web forward from marc thiele on Vimeo.

    Here’s what is covered in the talk:

    1. Modern web technologies of HTML5 and friends that can be used right now (with 64 myself as a demo) [03:50 - 07:19]
      • Rich HTML semantics (HTML5)
      • Self-validating forms (HTML5)
      • Richer form controls with fallbacks (HTML5)
      • Canvas for painting in the browser (HTML5)
      • CSS gradients, multiple backgrounds, animation and transition
      • CSS 3D transforms
      • Local storage and offline storage
      • SVG for scalable and interactive graphics
      • RequestAnimationFrame for secure animation
      • History API
      • WebGL
    2. Taking on challenges – we need you to show the world that web technology is good enough to do jobs that in the past were only possible with native or server-side code (with Joe Lambert’s image unshredder as the example) [07:20 - 08:04]
    3. Breaking the browser mould – showing that the browser interfaces can be manipulated with HTML and JS (with browser menus, context menus and the Fullscreen API as examples) [08:05 - 10:54]
    4. Developing with the web – developer tools in browsers and done in client side technologies (with Hackasaurus, Cloud 9 IDE, Firefox Scratchpad, Firefox Style Editor, Parse error display in view-source and Tilt as examples) [10:55 - 17:24]
    5. Online identity and issues with current login systems (with BrowserID as a solution to a lot of the problems we have right now) [17:25 - 29:35]
    6. Apps, the shortcomings and myths of native apps and the opportunity to build hybrid apps with web technologies (with Open Web Apps and Web Intents as examples) [29:36 - 37:34]
    7. Moving web technologies into the mobile space (with Are we mobile yet? and Boot to Gecko as demos) [37:35 - 40:11]
    8. How can you help? [40:12 - 42:33]
  3. People of HTML5: Joe Lambert unshredding images in Canvas

    Joe LambertToday we have a quickie for you: Joe Lambert, a web developer from Southampton, England working for Rareloop took on the Instagram engineering challenge of un-shredding a shredded image but instead of using server-side code, he used HTML5 canvas. Here’s a screencast of his solution to the problem:

    And here we are in a quick interview, chatting about how he approached the problem and why he used web technologies:

    1) The specs of the competition said you can use Python, Ruby or C++ and you went instead for JavaScript and Canvas. Why?

    JavaScript is my language of choice and what I use most of the time. I also find it the most fun as its easy to post your creations online and show people immediately, no downloads or dependencies to worry about. I wasn’t really looking to get hired by Instagram so adhering strictly to the rules wasn’t a big deal. Essentially it was an appealing problem that I thought could be solved in a browser, so wanted to prove that was the case.

    I actually knew from the start that creating the algorithm to unshred the image was only half of what I wanted to do. That in itself was going to be a challenge and quite satisfying to solve but I really wanted to get to a point where I could make the slices move into the right place. I wouldn’t have been able to create this effect as quickly in any other environment.

    2) How did you approach the problem of unshredding the images? Did you do similar things before? What is the logic you use?

    I wasn’t too worried about the most efficient algorithm so I just started by thinking about how a human might solve this type of problem. I’ve played around with image data a little before but nothing that would have been directly useful to this particular problem.

    If I were solving this in the physical world I’d start by picking up a slice at random then pick up each other shredded slice and see if it looked like it fitted to the left or right of the piece in my hand. I’d then repeat this till the image was back to its original state.

    This gave an indication of the kind of structure my code should have, I just needed to work out how to computationally measure whether two strips ought to be next to one another. For this I just compared the pixels closest to each edge and measured the euclidean distance between the colour values. It seemed to work pretty well!

    3) I see you are not using typed arrays in your solution yet. It seems to perform well the way it is now, do you think they’d make a difference?

    I’m not sure, efficiency wasn’t high on my priorities for this challenge so I didn’t really look to optimise it but as you mention it seems to perform quite well. I haven’t tried the algorithm with larger images but I suspect it would perform not so well.

    4) What was the feedback so far? Has Instagram contacted you?

    The feedback has been really positive, especially via Twitter. I think generally the competition has had quite a high profile which certainly helped.

    Instagram haven’t been in contact yet but it would be good to hear from them. I suspect they’re focusing on all the people who used the recommended languages

    5) Seeing how easy this was can you see other challenges with images and canvas? Could you think of ways how to make it easier for developers, for example by extending the API?

    The way you access pixel data in Canvas is a little complicated, it might be useful to have some helper functions to let you get access to a single pixel via X/Y coordinates. I ended up writing a little helper function to do this for this challenge, I’m going to try get the code up on GitHub shortly so this might be of use for others.

    It would also have been useful to have been able to call toDataURL() on a CanvasPixelArray object. My implementation used the DOM and CSS3 transitions to move the images into their correct places after the algorithm had solved the order, being able to access a Data URI for each slice would have be handy.

    If you want to chat with Joe, he is available on Twitter as @joelambert.

  4. People of HTML5 – Divya Manian

    HTML5 needs spokespeople to work. There are a lot of people out there who took on this role, and here at Mozilla we thought it is a good idea to introduce some of them to you with a series of interviews and short videos. The format is simple – we send the experts 10 questions to answer and then do a quick video interview to let them introduce themselves and ask for more detail on some of their answers.

    Divya Manian Today we are featuring Divya Manian, Web Opener at Opera Software.

    Most likely you came across Divya because of her involvement in HTML5 readiness and HTML5 Boilerplate. She is available on Twitter as @divya and is very much involved in the CSS standards working group.

    As you will see in the interview, Divya is a very pragmatic person when it comes to web standards and has a big passion for educating developers instead of woo-ing them.

    The video interview

    You can see the video on any HTML5 enabled device here (courtesy of vid.ly).

    10 Questions for Divya Manian

    1) I feel that right now is a terribly exciting time to be a web developer. Would you agree? What gets you really excited about the new tech we have to play with?

    Definitely. These are exciting times for a web developer. You have new tools to work with almost every other week and what your page is capable of doing has expanded significantly from just delivering static content to enabling real-time media streaming and more. We also have very strong developer tools for each browser: Opera’s Dragonfly, Chrome’s Developer Tools, IE Developer tools in addition to the original trail blazer Firebug. So, it is simultaneously easier and harder to develop for the web.

    2) Back in the days we very much preached separation of concerns as the right way to build web products. HTML is for structure, CSS for look and feel and JavaScript for behaviour. It seems to me that with new technologies this strict separation is blurring a bit. We can generated content in CSS and animate and transform. Some HTML5 elements do nothing without JavaScript (canvas being the big example). Do you think we need to re-visit our best practices?

    Yes, the new features do definitely make you think harder about what you put where, but they do enable a similar separation still, except there are nuances to be aware of when you do the separation.

    For example, I would consider animation with JavaScript as a way around the roadblock of not being able to do natively. Browsers are in a better position to control most of the animations that we require (animations for gaming are slightly different ballgame), and doing them natively would get us better performance in the long run.

    Personally, for me maintainability and writing something readable and that works well and efficiently is more important than just being driven to compartmentalise your code into HTML/CSS/JS.

    And definitely we do need to revisit our best practices every so often because technologies change and our best practices need to change with them, none of them are enshrined in stone and we should keep them relevant to our current set of features/technologies.

    3) In your post “This revolution needs new revolutionaries” you point out that a lot of the people who drive the web today are not the known luminaries of web design. Do you see a complacency in our advocacy of web standards?

    There are two concerns for me in that post:

    1. we are not hearing enough from people who have to deal with creating web applications that work in areas with poor internet connections, censorship, with content in languages that are not popular.

    2. There are not enough do-ers who are talking, we are hearing from the same people again and again on similar topics.

    I feel strongly about both, but more so about #2 because it impacts every web developer all over the world. There are changes that are occurring that most of them are unaware of, because word does not get out. I think we should do our best to encourage those who do actively seek to create tools, and help fellow web developers or work on interesting challenges for the web to speak and inspire rather than those who are known for their speaking abilities because ultimately we want people to use/work with what is best for the web and not just be informed of what was news 5 years ago.

    4) I found lately that collaboration is getting easier and easier. Tools like github, JSBin and JSFiddle allow you to talk about code and get your readers to fix and change things with you. I did that lately with getting bullet proof 90 degrees turned headlines for example. Why do you think not that many take advantage of that opportunity?

    I would be hesitant to say not many are taking advantage of these tools. They are, but it is true that not everyone is on the bandwagon yet. Github is certainly the most gentle and social introduction to version control you can get, but a lot of web developers are not programmers and have not seen enough pain and horror to know why version control systems are useful. It also requires knowing about what version control systems are, and how to use the command line (a bit), which might be scary for those who are just used to designing with IDEs or TextMate.

    5) CSS seems to be moving in leaps and bounds right now. I for one am very excited about the CSS element property which allows to take screenshots of elements. Are there any lesser known extensions you are fond of and use?

    I am not a big fan of vendor prefixes, and would rather see them quickly unprefixed rather than see more of the prefixes populating stylesheets more and more.

    That said, I do like a lot of the new properties that we are experimenting with. We have the obscure tab-size which allows you to control the width of the ‘tab’ character in your content. Pretty useful when you are displaying code.

    Opera also introduced the @viewport which will let you set the viewport from within your CSS rather than using meta tags (like <meta name=”viewport” content=”width=320″>). I think viewport belongs to CSS rather than markup, so I would love to see it gain adoption.

    Some of the less known properties such as box-sizing (unprefixed in Opera, IE 8+, Safari 5.1+, Chrome, prefixed in Firefox) are also invaluable, as they let you control your box model, which is definitely a revolutionary step from the dark days of trying to work with separate box models.

    6) When we had a longer chat before this interview we discussed that there seems to be a disconnect between what people show on stage at conferences and what people can use these days in their day-to-day jobs. Do you think we should remedy this? More hands-on stuff for people to use now rather than a “look what is possible” approach?

    I think what gets shown on stage is partly entertainment and partly information. I think it is hard to show “real hands-on” stuff without diving deep into it and losing half the audience while doing so. We need a balance for sure.

    7) When writing CSS these days I get very annoyed about having to repeat a lot of code with different browser prefixes. Animations are the worst with all keyframes having to be repeated. Do you use any preprocessors like SASS or LESS? What do you think of that approach?

    Yes, I was/am an early fan of Sass. I have been using it for 2.5 years (a lot less now as I do not deal with as much CSS as I would like to). I certainly think Sass/LESS would be the way to go forward for any web developer right now. They make CSS a lot more powerful and attempt to bring in programming paradigms that CSS sorely lacks. Attempts are being made by Tab Atkins at Google to bring these in a form of a proposal to the CSS WG, and hopefully we should see some form of support in the browser.

    I would definitely recommend doing it on the server side though, doing JIT compiling of such code would be such a disaster for performance.
    Especially today with so many vendor-prefixed extensions, not using such preprocessors would only cause more harm than not.

    8) You work for Opera, the browser that did implement the most of the HTML5 form elements. Why do you think others are reluctant to do the same? Do you think HTML5 forms are ready for prime time yet?

    It is certainly not true that others are reluctant. Chrome is pretty close to Opera in terms of support, and Firefox and IE10 are have various levels of support too. Yes, HTML5 forms need to be used with polyfills as of the moment, but I cannot wait for full support to land on all browsers so we can get beyond validating forms on the server.

    9) I get a feeling that there is a general fatigue of semantic matters in the HTML5 world. Showcases have no HTML at all or meaningless elements like DIVs as buttons and so on. Is it just not sexy enough when we can rotate things in 3D and make sounds?

    I am tired of semantics, too, really :) I think there is more to HTML5 than discussing when to use a section or a div or an article or an aside. Semantics are good to know about and learn to use, but we have had 15 years of talk about semantics surely we can go beyond that and learn about all the new stuff that occurs in HTML5 that will allow faster/more performant way to provide better experiences for your users.

    10) If you have a friend who just wants to start with web development, what would you tell them to do and go to? What is the most efficient way to get people hit the ground running these days?

    I would ask them to first hit the Opera Web Curriculum which has now moved to W3C – it is a wiki now so everyone is welcome to contribute to keep it up-to-date and relevant. Then I would highly recommend they refer to explanations and tutorials at the Mozilla Developer Center!

    Photo by Chris Casciano

    Do you know anyone I should interview for “People of HTML5″? Tell me on Twitter: @codepo8

  5. People of HTML5 – John Allsopp

    HTML5 needs spokespeople to work. There are a lot of people out there who took on this role, and here at Mozilla we thought it is a good idea to introduce some of them to you with a series of interviews and short videos. The format is simple – we send the experts 10 questions to answer and then do a quick video interview to let them introduce themselves and ask for more detail on some of their answers.

    John Allsopp Today we are featuring John Allsopp, author of Developing with Web Standards and organiser of the Webdirections conferences.

    As you will see in the video, John is neither mincing his words nor does he hold back in spreading good messages about the web as a whole and independent of technology. He has been around since connecting to the web meant enduring bleeping noises and has spent quite some time building software and online generators to allow people to build for the web.

    He has been very outspoken about the lack of semantic improvement in HTML5 and you can count on him to speak up every time somebody claims that native apps on mobiles will always be better than web standards based apps.

    The video interview

    You can see the video on any HTML5 enabled device here (courtesy of vid.ly).

    10 Questions for John Allsopp

    1) You’ve been around this internet thing quite a while and seen a lot of things come and go. What makes you believe in HTML5?

    Well, interestingly I’m probably known by some people for being the anti-HTML5 guy. I’ve been quite critical in particular of the semantics in HTML5, both about some of the specific decisions made, but more importantly the approach to extending semantics in HTML5.

    My particular interest in HTML5 in the broadest sense (so, including CSS3, related extensions to the DOM, geolocation, DAP etc) is seeing the web become an increasingly powerful application platform.

    Over the last 20 years or so, we’ve seen the web revolutionise publishing, journalism, and other essentially text based media. This emerging generation of technologies promise to revolutionise much more.

    2) What are the things in HTML5 that still need tweaking in your book? Anything that ails you about the current state of the spec?

    Well, I still think the ball has been dropped when it comes to increasing the semantic richness and expressibility of the language in any genuinely useful way. And I sadly don’t really see anything happening there.

    I also think HTML needs to take a leaf from the CSS book – CSS2, a monolithic specification became increasingly bogged down in minutiae, and the failure to finalise it became a kind of running joke.
    With CSS3, we saw the modularisation of the specification, with much more rapid innovation, much more experimentation. In truth, it was the experimental CSS3 features that I think ignited the excitement around what we call HTML5, far more than a new DOCTYPE and things like the header element.

    The core of the HTML5 specification is really monolithic, and is suffering much the same way CSS2 did. We’re even seeing the same sort of glib jokes about how it’s going to be finished in 2020, which impacts on whether many developers will adopt something perceived (mistakenly, though somewhat understandably) as unready for commercial use. Some aspects of the initial monolithic HTML5 spec have been modularised, while related technologies like geolocation, which is widely considered to be part of HTML5 have always been their own small, independent activities.

    So, if anything, I’d like to se the increasing modularisation of the HTML5 spec.

    3) You seem to spend quite some time building code generators at the moment. The CSS gradient generator and the audio embed generator being just two of them. Do people use them a lot? And why the separation between Firefox and Webkit rather than generating both?

    Just to clarify, all my new tools (audio, video, and some others in the pipeline), as well as the upgrades to existing ones (linear and radial gradients) are cross browser – they work in all browsers that support the technology they’re focussed on, and create code for all modern browsers (in some cases where they don’t even yet support the particular functionality – for example Opera and radial gradients).

    In the past (and some of these are 2-3 years old now), I decided to focus only on those browsers which supported a particular technology. I had separate Firefox and Webkit versions of the gradient tools because they supported quite radically different models of gradients, which made a single interface to producing two separate versions of a gradient confusing, as well as essentially impossible.

    It’s important to note that the primary motivation for these tools is to help people learn the concepts and syntax for these various features. I think many CSS3 features in particular are really quite complex, and learning them in the more traditional text oriented way is becoming increasingly difficult. For example, you can generate a radial or linear gradient with my tools for all modern browsers in a few seconds, even if you have no idea about how CSS gradients work. But the two articles I wrote on gradients in total come to nearly 10,000 words, and a couple of dozen or more illustrations.

    Do many people use them? – they certainly generate quite a bit of traffic. They’re definitely popular – and I think as importantly, people really appreciate them – even really well known CSS experts let me know they really appreciate having them there, and I know others are using them to actually test browser implementations. So I think they are well worth doing.

    The last thing I’ll say on them is I also build them to let *me* learn – I’ve been developing tools for web developers for 15 years, all the way back to the earliest days of CSS, and I find the best way to know a technology (and its and your limitations) is to build a tool to work with it.

    4) You are also a conference organiser and run workshops. What are the current hot topics in those related to new, open web technology and are there things you’d like to cover more that are not yet requested?

    New CSS3 features definitely get a lot of interest – animations, shiny effects like gradients and so on. And we’re definitely seeing a lot more interest across the board in the app development related technologies – offline webapps, geolocation, JavaScript more broadly. Im a sense I think a lot of more traditional web design is “done”. There are widely accepted and understood markup practices, patterns and techniques. There are page architecture patterns that we increasingly build page and site designs out of. But the area of web application development is really uncharted territory by comparison.

    5) A sure-fire way to tick you off and get you to blog seems to be to say that native experiences on devices will always be better than using open web technology. If you look at it straight from an end user perspective, this seems to be the case though – things look smoother in native apps. How can we make standards-based development more interesting for developers when the lure of native apps with a money-making store is that big? How can we break the notion of open tech apps being of lesser quality?

    There’s a lot going on with this question, so let me begin by stating my basic case. I call BS on the argument that native apps are intrinsically better looking, have better UX, and so on, simply because they are built with say objective-c (when people say native, they almost invariably mean apps for iOS they *think* are built with cocoa touch).

    Your observation that many “native” apps are slicker, etc than web apps is a fair one. I think that it *is* somewhat easier to create a cookie cutter native app for say iOS using CocoaTouch, and the really excellent tools Apple has been working on since the NeXT days. But it’s also important to note that for many categories of applications, for example games, these apps use technologies like Unity3D, so in a sense many of the most successful, most engaging applications people use on say iOS aren’t native. I think it really comes down to the tools – and for developing apps using web technologies, these are far less sophisticated on the whole than those like Apple’s. Which to me presents an opportunity.

    As to the lure of big money – it’s again one of those myths that get repeated ad nauseam. In fact it started even before there were any apps in Apple App Store! Now we’ve seen several years of activity, what have wen learned? There are doubtless a small number of big winners – but all indications are that almost no one makes any more selling apps on app stores.

    There’s a whole industry in app discovery, so the supposed bon to developers that the App Store was going to provide in allowing the little guys to get their apps discovered by users really isn’t panning out.

    As soon as you start looking at any sort of numbers objectively, it’s hard to see business case for building a native iOS app and selling it on Apple’s App Store, compared with simply taking that money to Las Vegas for the weekend. You’re going to waste far less of your life.

    Finally, the single best way we can address the challenge web technology based apps have of being perceived of inferior quality is to continue building apps that are of superior quality. There are plenty out there – and often you don’t know it, because they are packaged up as native apps.

    6) I am worried about showcases these days. The latest “Chrome experiment” (the collaboration interactive video with OK Go) didn’t even work on my Macbook Air with Chrome and let my fan do the impression of an aircraft taking off (it also, ironically, doesn’t work on my Chromebook). There seems to be a – to me – dangerous move towards experimental web sites expecting certain browsers and even hardware. Are we facing a new “best viewed with IE6 and 1024×786 resolution”?

    I understand the concern, but don’t think we’re seeing, or will see a return to those bad old days. The “best viewed in” was seen on many many production sites back in the day. Experiments are just that – something at the bleeding edge, pushing the technology to the edges of its capability. So, I love seeing these experiments. We’ve been involved in some ourselves. We typically kick off our conferences with an opening title sequence (we started this a long time ago, but we’re seeing loads of folks doing it now). In the last year or so, we’ve been commissioning people to create these sequences using web technologies – and we’ve got no trouble with them targeting even a really, really specific setup – after all, it’s being designed for a very specific circumstance.

    I honestly don’t think we’ll head back to the best viewed in days. Developers and browser makers have learned the lesson of just how bad that is for everyone.

    7) Whilst there is a lot of uptake on cool CSS3 transitions, animations, 3D transformations, WebGL and video it seems to me that the semantics of HTML5 and the forms get a bit forgotten. Even more so, it seems like a scam to tell people that there is a need for sections, articles and time elements when no browser uses them to create an outline or syndicates them. Do you see that changing? How can we make it more real for people? Haven’t we used the “use this and that markup as it is the right thing to do” argument too often?

    I tend to think the use of various new HTML5 elements as something of a cargo cult. I don’t really see any practical benefit, and there’s still legacy browser challenges (IE6 and 7) to deal with there. And I doubt there’s ever really going to be particular benefits – there’s ultimately very little additional information we’re providing by saying “this is a section”, “this is navigation”, and so on. If the semantics were richer, there would potentially be much more browser developers, search engine developers and so on could do with the richer markup, but as I’ve mentioned, I think we’ve got what we’ve got.

    As for forms, now, there’s a lot more going on here of practical use today, and into the future. We can use new elements like number, email and url, and attributes like required right now (I’m doing it myself), as these fallback nicely in older browsers.

    In fact, it’s one of my next big areas of focus after I finish a little project involving animations! HTML5 forms stuff is awesome!

    8) Let’s talk accessibility a bit. I get the feeling that right now the accessibility world is falling immensely behind. Whilst HTML5 and jQuery tells people to “write less and achieve more” it seems that ARIA roles are ridiculously verbose and rather tough to add. A lot of the needs of mobile and touch interfaces overlap with the needs of disabilities. However, it seems to me there is not enough interest in the a11y world for new technology as it is not proven. Do you find it hard to connect the two?

    I’m very far from an accessibility expert, so there’s probably not a a great deal I can add to this particular area of any real value.

    ARIA roles are how I’d actually go about enriching the semantics of HTML5, rather than adding a small number of new elements as is the current state of play.
    There’s two aspects to ARIA roles. First is the role attribute, initially introduced in XHTML as a means for adding additional semantic information about an element – the role that element is playing. For example, we have a common markup pattern of a div element, playing the role of the page header. We can mark this up with the role attribute like so <div role=”header”>

    role is actually quite separate to ARIA. What ARIA brings to the role attribute is a vocabulary, a number of possible values for role, that we can use to add further semantic information to our pages. But you aren’t restricted to ARIAs vocabulary, and indeed can define your own. ARIAs vocabulary is actually far more extensive than HTML5′s.

    I think the markup pattern, of using an attribute with a value to give additional information about an element, is really well known and widely used by developers – who doesn’t use class more or less like this?

    So, I don’t see the use of role as a particular challenge for developers to adopt.
    Then, role has the advantage that you can style it in every browser from IE7 up, using CSS with no JavaScript as well. For HTML5 elements, we need JavaScript for IE7 as well as IE6.

    It has to be noted that he ARIA vocabulary is a generic one for rich internet applications, not specifically designed for apps developed using HTML. And sadly the ARIA vocabulary, and HTML5 semantics don’t map onto each other well.

    I guess the biggest challenge for web accessibility into the future is that while making essentially text based and static content accessible is not overly challenging, and is now well understood, making applications, dynamic media, and rich interactive content accessible is a far more significant challenge.

    9) To me it is high time we stop doing showcases and demos and instead start documenting what people can do themselves and build real products with open web technologies.

    I love the showcases, the demos, the proofs of concept, but I agree, I also think it is time to really draw attention to the real world examples. I guess with my articles and tools, what I’m trying to do is help developers adopt these technologies today. As are many other folk as well.

    9) (ctd) Sadly, when it comes to conferences though a lot of the things shown on stage were written as a demo for the talk. Are there any exceptions you found? I remember some @media had a great talk on how Blogger saved money by moving to CSS for layout. We need more hard evidence talks like that. Do you actively try to find real life showcases?

    Absolutely – one of our goals is to have people who walk the walk. We actively look for folks who go beyond the opinions, and can speak with experience about projects, showcase successes, talk about the challenges they face, and the solutions to those.

    10) What’s the next big challenge? Where would you like the web to go and what are the next things that browser makers should agree on?

    Those of us old enough will remember when suddenly everything had an LED then LCD screen – things that were once analog, with mechanical dials and the like, things like microwave ovens, suddenly had displays, for time, temperature settings, and so on.

    I think far sooner than we realise, there’ll be high resolution touchscreens everywhere, on almost every device.

    Not all these devices will necessarily be connected to the web, but many of them will be. But their UIs will be essentially browser technology.
    So, there’s going to be huge opportunity to develop user experiences for all kinds of devices using HTML, CSS, JavaScript and related web technologies.

    Another really significant trend we’ve seen faltering steps toward, but which I think will genuinely be a paradigmatic change in technology use is “boot to web”.

    Think back 3 or 4 years. The hardware, chip, motherboard, RAM speed of your laptop was a big deal. People really cared. They bragged about this stuff! Apple went through 2 incredibly complex expensive CPU architecture changes over about 12 years because of it. Now virtually no one cares. Hardware doesn’t matter any more. It’s been commodified.
    The next layer to be commodified will be the OS. It may seem unimaginable now with iOS so dominant in the mobile, not to mention tablet space, but to me this is a very transitional period. The web will commodify operating systems. ChromeOS is a commercial example of this. Boot to Gecko, something Mozilla IMO should have been working on a long time ago is a more experimental example.

    The technologies we currently call HTML5 will be the building blocks of the applications which run on these devices.

    The as yet missing piece of this puzzle is the emerging Devices API standard, which exposes hardware and system APIs like the camera, address book, calendar, messaging services (for example SMS) to the browser. We’re already seeing support for aspects of this, such as the camera in Android 3 devices. I’m sure this is a big part of the “Boot to Gecko” project too.

    Photo by Sam Whiteman / TEDxNSW

    Do you know anyone I should interview for “People of HTML5″? Tell me on Twitter: @codepo8

  6. People of HTML5: Mr. Doob

    HTML5 needs spokespeople to work. There are a lot of people out there who took on this role, and here at Mozilla we thought it is a good idea to introduce some of them to you with a series of interviews and short videos. The format is simple – we send the experts 10 questions to answer and then do a quick video interview to let them introduce themselves and ask for more detail on some of their answers.

    Mr. Doob Today we feature the elusive Mr Doob. Ricardo is a man that always crops up when the talk is about impressive technology demos. Starting with the ball pool, continuing with the beautiful Harmony and culminating in The Wilderness Downtown and Three Dreams of Black he continuously got commissioned to do things Google showed off at their big events.

    Ricardo values his privacy, so he didn’t want to be interviewed on camera. We respect this, and instead only give you his answers in text.

    1) Welcome. It seems to me there is not a single conference or article about the use of modern web technologies that doesn’t feature work by you. You also seem to have appeared out of nowhere. What’s the secret? What is your background?

    My brother introduced me to the demoscene at the age of 12 and slowly learnt my way creating graphics and modelling lowpoly 3d scenes to be rendered in realtime. So far I’ve done more than 50 demos in my spare time. Only on the last demos I did the code part though. My role was always being the designer, illustrator, 3d modeller and/or director.

    During that time I was working as HTML/PHP developer and later, when I moved to London, I started working as Designer/Flash developer. At this point I was slowly becoming a graphics programmer and I saw the chance of messing with the tricks I saw being used in all these demos and also finally putting into practice my own ideas for effects. I started uploading the experiments on the mrdoob website with no real reason. That turned out to be a great way for attracting interesting projects that allowed me to keep experimenting.

    2) A lot of the work you’ve done is experimental and very much on the bleeding edge of what browser technologies can do. Live products on the web however should not frustrate users and tell them to upgrade their browser or give them a broken interface. What is your stance on progressive enhancement and polyfills for older environments?

    Well, the demoscene is way worse. If you don’t have the newest graphics card, chances are you can’t run the new demos being released. You have to upgrade your graphics card every few months. The browser world is simpler, you just need to download a piece of software which you don’t even have to pay for. And, now that browsers seem to be adopting auto-update mechanisms, feels like this problem will finally go away. However, I always develop on fairly low-end computers (MacMini/MacBook) so I can avoid the demoscene problem.

    As per progressive enhancement… if I had to include some kind of backwards support in the experimentation process the result will for sure get affected.

    Not only the code wouldn’t be as simple but, in some cases, I wouldn’t even start because it wouldn’t be as fun. With my current approach, some users won’t see anything or get errors, but there is a chance that some of these will upgrade to a modern browser in order to play with the experiment. Which, not only helps me, but the rest of developers and, I want to think, also the whole web.

    3) Whenever a new demo comes out and it only works in a certain browser without gracefully giving others a diminished experience I get the feeling we are not doing a good job advocating open technologies. After all no clothing store would display dresses in edge case sizes or with missing buttons. Do you agree? Or is it OK to build a demo for one single environment and call it a “web technology showcase”?

    I personally try to make sure all the projects I do work on as many modern browsers as possible and I try to avoid technologies that have only been implemented in just one vendor. There are cases, however, that a experiment may not work in a browser because the Javascript engine isn’t fast enough.

    These demos make the problem evident and if the browser developers notice it they’ll work to make sure their engine is fast enough to match the other browsers with that demo.

    4) Looking at similar groundbreaking work like you’ve done in the demo scene I found that the really successful groups tend to build their own tools for edge case tasks. When Farbrausch for example released their 96KB 3D Shooter game .kkrieger they soon after also released werkkzeug to create similar games and animations easily. Do you have some awesome tools up your sleeve? What is your development environment?

    No tool yet. But now that the Filesystem API is being adopted I’m starting to seriously consider it.My development environment is, for some, surprisingly (and disappointingly) simple. I use Ubuntu’s default text editor gedit with a dictionary-based code completion plugin. I also have a plugin enabled that shows unneeded spaces at the end of the lines – I’m a bit obsessed with clean and spaced code.

    5) What product are you most proud of and why? Who inspires you? Are there any cool web technology demos that made you go “wow”?

    I don’t know why but I’m unable to be proud of anything I do. There is always something to improve, and that thinking makes me almost be ashamed when talking about my work.I get inspired by anyone that spends an admirable amount of time in creating something different or unique. Thankfully, the list of names would be endless. Also, open source as a whole inspires me, not only libraries but the linux world in general. The fact that you can see the code other people are committing pushes you to continue doing your part. As per impressive technology demos, this Path Tracer and the latest experiments by the miaumiau guys are good examples. I wish I knew how to do these things :)

    6) Are there parts of the technology implementation of the HTML5 standard do you still consider terribly broken? For example the web version of Angry Birds uses Flash for the audio – why is that?

    I won’t get tired of saying that <canvas> should bring the “darker” blending mode back…I haven’t played with <audio> much yet, but I haven’t heard good things about it. Hopefully the upcoming Audio APIs will sort this out. They first need to agree on an API though.

    7) A lot of the “wow” that we are shown these days in open technologies has been done in Flash before. You hear a lot “this is not Flash, even when it looks like it”. Are we just reverse engineering what the closed technology world did in the past or is there some re-use and knowledge sharing going on?

    And a lot of the “wow” that was done in Flash was done before in MSDOS and Amiga (yes, sadly the gap was that big). But we managed to bring to the platform some of the tricks that the Amiga people found during all these years. With WebGL now the web is finally catching up with modern graphic pipelines. The problem will now be that website production will become more costly. A bit like the videogame world where they spend many months (or years) in production.

    8) Let’s quickly talk about performance. When I watch 3 Dreams of Black on my MacBook Air it sounds like my vacuum cleaner and the battery drains very quickly. What are the most resource hungry technologies in the open stack and what could be done about this? Will there be adaptive technologies that for example give a higher poly version for beefy machines and low poly versions for slower ones?

    With video we have something like that in Silverlight and Flash when it comes to streaming. Is there work done in open technology we can look forward to?There is a tiny detail to consider there, I used to have a MacBook Pro with an ATI graphics card and the fan would get noisy the second it started to render a few polygons. Now I use a MacBook with a NVIDIA and I hardly hear the fan. The GPU use is the same, but just happens that the graphics card was designed to be quieter. If you happen to have such a card the performance issue decreases considerable – as there is no annoying noise.

    However, we should indeed optimise our code and only render what’s required when it’s needed. As soon as WebGL lands in Android/iOS devices we’ll probably start to seriously considering serving low-poly objects to such devices.

    9) Resources. Let’s say some gifted kid coming from school right now wants to be Mr.Doob and bring the awesome. Where can you learn? What should you concentrate on?

    Not sure… I guess one needs to spend a lot of time tinkering with code, experimenting and creating projects just for fun. While you’re in such a loop ideas arise constantly and the only thing left is spending more and more time turning these ideas into life.

    And now that we’re moving to HTML5/Javascript, you only need to right click and view the source. I still can’t even begin to imagine the ramifications of this simple fact.

    10) What’s next? Anything cool coming our way we can look forward to? Also, what do you want? What do you think keeps us from going really all in when it comes to open web technology?

    Well, right now I’m still trying to “solve” three.js. It does feel like an endless task right now, but I’m confident there is a way. With libraries like stats.js, tween.js and sequencer.js I have proven myself that libraries can reach a state when do not require more features.

    Lately I’m on a personal crusade on creating open source libraries and tools that make the web development possible without the need of proprietary tools and operating systems. I have a ton of .psd and .fla files that I can’t open now that I run Linux. I would like to avoid such scenarios to happen again.

  7. People of HTML5 – Seb Lee Delisle

    HTML5 needs spokespeople to work. There are a lot of people out there who took on this role, and here at Mozilla we thought it is a good idea to introduce some of them to you with a series of interviews and short videos. The format is simple – we send the experts 10 questions to answer and then do a quick video interview to let them introduce themselves and ask for more detail on some of their answers.

    Seb Today we are taking a bit of a detour from HTML5 as we are talking to Seb Lee Delisle. Seb is a long-standing, award-winning Flash developer who lately focuses on “creative coding” independent of platform and technology.

    In his workshops he makes web developers leave their comfort zone of defined and limited environments and shows what can be achieved if you play with technology rather than feeling hindered by it.

    I’ve invited Seb as I see him as a great opportunity to learn from what the Flash world has been doing for years – in a lot of cases they already overcame problems the HTML5 world has yet to face.

    The video interview

    You can see the video on any HTML5 enabled device here (courtesy of vid.ly).

    Ten questions about HTML5 for Seb Lee Delisle

    1) You are very known to give workshops and talk about very visual programming – creating beautiful, and highly technical stuff. How do people normally react and do you think we will see more of that in the mainstream?

    Generally people seem to react really well! Particularly in the JavaScript community where people are now starting to discover the new creative possibilities in this medium. The part I love the most is showing how such simple code can create beautiful effects.

    I’m already seeing subtle animated graphics crossing over into the mainstream – just look at what’s happening with the Google doodle lately. My job is to help programmers to develop a sensibility for animation and use it tastefully.

    2) In the past you’ve been known as a pure Flash guy – and actually one of the few that released code outside the Flash community and tried to cross boundaries. Personally I am so tired of the “HTML5 as the Flash killer” and “Flash as the saviour of the web as it is truly cross-platform” arguments. How can we do more to break the barriers between the Flash and non-Flash communities?

    Although known primarily for my (BAFTA winning ;-) ) Flash work, these days I seem to be growing into a polymath! In the last month I’ve worked in C++, Processing, JavaScript, and ActionScript and enjoy switching between them all.

    The most important thing is to learn as much as we can about all technologies. Then you can make informed decisions about the best tool for the job.

    I’m pleased to see other famous Flashers doing great work in JS. Mr Doob (of three.js) used to be a Flash programmer and is an amazing visual coder. Grant Skinner is making a canvas graphics library called Easel.js.

    I think it’s unlikely that JSers will be keen to invest time into learning Flash, but it’s important to recognise the strengths of the plug-in and use it where appropriate.

    3) Looking at the open web stack, what gets you most excited? Canvas? SVG? Processing? And why?

    Canvas is fun but can be a bit CPU heavy if not treated with care. SVG is very interesting, particularly Raphael.js which is a library that neatly wraps SVG and falls back to VML – it even works in IE6! SVG is definitely under-used, particularly when IE9 can render it super fast with the GPU. But we’re lacking good tools to make animated content.

    I love the Processing community and Processing.js makes things quite accessible, but for truly optimised canvas rendering you’re probably better off learning the native canvas API.

    Canvas is the new thing so naturally everyone’s really excited about it but I think there’s still lots to explore in just moving DOM objects around. You get better backwards compatibility and often it’s faster than canvas.

    4) What are the limits of open web technology? Where would you see the next phase of research into standards to go?

    This is a really hard question to answer! I guess the biggest difficulty is the sheer number of different vendors that have to work together (or not ;-) ). But the proliferation of webkit means that we’re seeing browsers pop up in a variety of different devices from your eReader to your TV. As these devices get faster and faster it’s going to be a realistic possibility to have games and rich content everywhere. Whether we want that or not is a different matter!

    Browsers are still far behind Flash in terms of video, camera access, socket connections and P2P. I would say the most important of these would be video. Browsers should just be able to play a video in the page. But this thorny codec issue doesn’t seem to be disappearing any time soon.

    5) Beautiful things also need to perform well. What are your main tips to create smooth animations and interactions that don’t flicker?

    This is a hugely wide ranging question! But it’s usually the rendering that’s the bottleneck, especially if you’re working on canvas. Only redraw pixels when you absolutely need to. I’ve just made a demo of the game Asteroids, where I move a little canvas around for the ship, rather than clearing a massive canvas every frame even when just a little bit of the screen needs updating.

    If you’re moving DOM objects around, use the 3D CSS transform – even if you just set a z position of 0 you get hardware acceleration in webkit browsers. I’ve made a particle system that handles hundreds of DOM objects and is super fast, even on iOS.

    And use requestAnimationFrame rather than a setInterval for your redraw loop. This ensures that you’ll sync with the monitor refresh rate avoiding screen tearing. Perhaps more importantly, it’s disabled when your browser tab isn’t visible which saves your battery.

    6) A lot of the things we do when building games and animations is cheating. I remember working on very low-spec environments like Commodore 64 where you could have used real physics algos but found something much less computation heavy that did the job and relied on the human eye to be lazy and fix glitches for me. Is that what we still do?

    It’s still what I do. There are very few resources that outline these easy solutions, most books are very advanced. I’ve accumulated and devised these techniques over the years which is why I love sharing and demystifying these techniques on my courses and presentations.

    7) Where web design is very much fluent and you never know about resolutions and available screen space (which is why you test for it before you open for example a menu) most gaming seems to be within fixed boundaries. Canvas for example has a fixed size in pixels. Is that a limitation? I always found the unknown part of webdesign the thing that excited me the most as it meant my code has to be bullet-proof.

    I agree, but of course this is a huge challenge if you’re building a game and you usually work to a fixed pixel size. Anything based around bitmaps is fixed although I guess you can scale them up and down. Even Flash games seem to use bitmaps these days. It’s ironic because Flash is very good at scaling vector graphics. Historically Flash games have been set to a fixed and small pixel size due to performance issues in the early FlashPlayer versions.

    8) Animations just didn’t look right before libraries started implementing easing to simulate natural movement with acceleration and slowing things down naturally. The next step of that is bringing physics into animations. Is that easy these days?

    There’s a technique that’s familiar to visual programmers – a real time ease-out that I like to call tweasing. In fact i made a library in Flash called Tweaser that wraps this technique. Traditional tweeners have a start and end point and a duration. Tweasing just has a target and the object eases towards it. If you keep a bit of momentum from the previous frame you can even implement a springy naturalistic movement.

    The best reason for using this is that you can change the target at any time during the animation and your object updates its trajectory in a smooth way. It’s really powerful and responsive, especially with 3D. Check tweaser.org for a full video explanation of the technique. Perhaps if enough people want it I’ll port it to JS!

    9) What do you think will be the killer use for WebGL on the internet? I’ve played with Google’s Body Browser and I loved it – do you think the web and the open stack is ready for 3D or is Flash with Molehill still ruling that sector?

    I’m not sure what the practical uses for in-browser 3D are yet. I expect we’ll probably see some pretty exciting 3D mapping from Google. But of course gaming is what everyone is excited about – it remains to be seen how these 3D gaming capabilities will affect the casual in-browser games market.

    There’s something very pleasing about WebGL enabling GPU graphics right there within your browser, but it’s a big problem that IE has no plans to support it. Flash’s GPU enabled player (codename Molehill) has a huge advantage – FlashPlayer take up rates have been incredible with new versions of the player reaching 90% penetration within a few months. I don’t think there’ll ever be another plug-in with the penetration rates of Flash.

    Both WebGL and Molehill have huge workflow issues, despite the open source libraries like three.js and Away3D. Thankfully Unity3D have announced that they’re targeting Molehill. Unity3D is truly a joy to work with – it seamlessly imports 3D models and bitmaps, has built in physics, lighting, shadows and a particle system. You even program it with a flavour of JavaScript! So if they could also target webGL…?

    10) Inspiration time. What are things you think people should look at to catch the bug of visual programming and where are the tutorials to learn about them?

    If you want code tips, you should look no further than Keith Peters. Keith is well known in the ActionScript world for his Making Things Move books. He’s just completed 30 JavaScript experiments in 30 days – well worth checking out.

    Outside of JavaScript I’m hugely influenced by digital artists such as Robert Hodgin (@flight404), Chris O’Shea, Jared Tarbell, and Jer Thorp (@blprnt), who all work in a variety of different programming languages.

    It’s a really exciting time for interactive technology and in addition to my CreativeJS workshops, I’ll be continuing more large scale motion detection projection projects in openFrameworks and Processing.

    Photo by Andy Polaine

    Do you know anyone I should interview for “People of HTML5″? Tell me on Twitter: @codepo8

  8. People of HTML5 – John Foliot

    HTML5 needs spokespeople to work. There are a lot of people out there who took on this role, and here at Mozilla we thought it is a good idea to introduce some of them to you with a series of interviews and short videos. The format is simple – we send the experts 10 questions to answer and then do a quick video interview to let them introduce themselves and ask for more detail on some of their answers.

    john foliot Today we are featuring John Foliot co-chair of the subcommittee on the accessibility of media elements in HTML5 and well-known accessibility warrior of the wild wild web.

    John’s been around the web, mailing lists and W3C meetings for longer than he’d care to remember and has always been a voice of reason when it comes to making the web accessible for everyone and still use new technologies. As this is a topic not often covered in the HTML5 context we thought it a good idea to bring up.

    The video interview

    You can see the video on any HTML5 enabled device here (courtesy of vid.ly).

    Ten questions about HTML5 for John Foliot

    1) There is quite some confusion going on about what HTML5 is. Even the W3C shot an own goal with the release of the HTML5 logo and describing it as a branding for modern development that encompassed other technologies. What is your definition of HTML5?

    Personally, HTML5 is the next major iteration of HTML – Hyper Text Markup Language – which will be standardized by the W3C in due course. It will effectively take over from HTML4.01/XHTML1.1, which has not been updated since December 1999. It is one of the core standards that support today’s Open Web Technologies.

    I understand and accept however that in terms of marketing and mass communication, having a term that describes the idea of the current larger eco-structure is by necessity going to be abstract. The conversation around naming has gone on for some time now, and I suspect that most developers remotely interested already know that the stack of Open Web Technologies we are using needs a name (my buddy Bruce Lawson dubbed it NEWT).

    But the horse was already out of the gate, and HTML5 is the name. So when communicating with developers in general today, I believe we all understand what is meant by HTML5. HTML5 has also become the way the mainstream thinks of what it is we are doing, so I really don’t worry too much about a marketing name. Over time, a new buzz word will emerge, the world will move on, and HTML5 will mean the standard for hyper text markup language – fifth iteration.

    2) There seems to be two battling sources of truth about HTML5. The WHATWG and the W3C. When would you consider which group to be more important? What is the relationship between them?

    Their relationship? Complicated.

    The relationship acknowledges that two diverse groups can share a common goal, and working together can lead to progress. But it is hard work.

    The WHAT WG serves a useful – likely critical – role in the emergence and growth of many of the Open Web Technologies. With the support of many (but not all) of the browser vendors, a common sharing of technical ideas, including experiments and evaluation by multiple teams, leads to improvements in the technology. It is the place of the engineers, and it is accustomed to working in an Agile environment, which works well for engineers. It’s an exciting place where new ideas emerge and first get roughly specc’d out, and then refined, seeing greater support in developmental browsers, and then reaching released browser versions.

    However, even if tentatively working, those specs may not yet be complete, even if being implemented; often implementation is varied from browser to browser, and shortcomings (sadly often with regard to accessibility) are often significant.

    The specification WHAT WG are working is constantly evolving – which is good – but it makes it very difficult for large entities outside of the browser vendors to keep up. Large scale projects require more stability than what a “living specification” can provide, as by its nature that specification is in constant change. Witness this recent comment on a developer’s blog:

    “The worse part of it is that you can’t simply search blogs/forums or look the documentation to figure out how to solve the problems, since “nobody” knows how to do those things, information gets outdated really fast with the release of a new OS version, documentations are very poor and doesn’t cover edge-cases, specs are constantly changing and different versions of iOS adopt a different set of rules…” – and that was just trying to get <video> to work properly on the iPad.

    As well, by organization and politics, not all vested interests are part of the WHAT WG process, which can be problematic but not insurmountable.

    Microsoft is not part of the WHAT WG, and likely never will be. We can rail on about the problems that IE has wrought upon us, but with a current majority market-share we simply cannot ignore it. So the WHAT WG is somewhat ham-strung by this situation.

    I am known as being a staunch W3C supporter, as I believe very strongly that they best carry the banner for Web Technologies on a global level. The W3C is more than just websites in browsers; their Standards cover a wide scope of related technologies that all leverage the larger network, and whenever possible the W3C strives to ensure that they can all work together. It is a global organization that has financial support and backing from national governments, academic institutions, and technology companies that rely on the global internet network. I acknowledge however that like any large entity, it can sometimes struggle with rapid movement – bureaucracy being a necessary evil in any globally reaching organization.

    For a younger person, involved in the daily race that is this new technology, this pace can be understandably frustrating, but engineers working in larger corporations such as IBM, Microsoft, etc. are accustomed to this already. Nobody has to like it, but it exists.

    The WHAT WG explores ideas and works out wrinkles, W3C stabilizes, scrutinizes and “publishes” a benchmark that everyone can refer to. Both roles are important, but each is unique.

    As to the “truth”, I will leave that to others to decide.

    3) You are quite an advocate of accessibility. Do you see the new open technologies as a benefit for accessibility or are we forgetting about
    it?

    I think much of what HTML5 is starting to deliver will be of benefit to all users, including those using Assistive Technology. However much of what is promised is not yet supported in all browsers, and related technologies – Assistive Technologies – have a long way to come to leverage this benefit.

    For example, most browser rendering engines do not do native processing of the landmark elements (I believe Firefox 4 uses an HTML5 rendering engine), and support for other parts of the emerging spec that have an impact on accessibility is still lacking. All the more reason to get HTML5 Standardized; certainly to Candidate Recommendation at the W3C.

    I think that some of the emergent stuff that the WHAT WG is working on occasionally falls short on accessibility concerns; thankfully less so now than when some of the earlier HTML5 work was undertaken.

    Overall, I am generally thankful to see how much the awareness for accessibility has grown, and the generally productive dialog between engineers and accessibility specialists.

    4) A lot of showcases of HTML5 show a certain effect but fall behind in basic accessibility concerns. There’s no keyboard interaction and a lack of testing for support. You find empty links pointing nowhere and HTML that is stored in the document for later use – like a whole list of error messages that get shown and hidden when needed. What could be done about this? What is your message to developers why keyboard access or even non-JavaScript fallbacks matter?

    I think part of the problem, perhaps the biggest part, is lack of education. The barrier to entry for creating and posting to the web is extremely low: any fool can do it, and so even fools do – without knowing or understanding what it is that they really are doing. It is also hurt by web-sites and blogs that recommend some of these poor techniques as “the way to do…”, further perpetuating bad coding practice.

    I think one of the other problems, especially with regard to accessibility issues, is that many developers (that have any kind of engineering background) have become very comfortable, even dependant on, the AGILE development process: code it, test it, break it, fix it, test it, break it, fix it,… I can’t count how many times I’ve heard “This is just our first iteration, we’ll get to the accessibility stuff in a future build.” The problem here is that accessibility is relegated to the role of ‘feature request’ instead of ‘core requirement’. Here again, it’s a larger education issue: mainstream developers need to understand and commit to making accessibility a core requirement, which then helps shape the functional and technical decisions that they will be making as their project evolves.

    Keyboard access is an important requirement for many users, and increasingly not just users with disabilities. The massive growth in web content being created for mobile devices is helping make developers more aware of this fact. I challenge everyone reading this piece who creates web content to remove the battery from their wireless mouse for one day, and then test their site(s). This is perhaps one of the easiest “accessibility tests” any developer can do, as it does not require any special tools (hardware or software) for testing, and yet has a huge impact on both non-sighted users as well as users with various forms of mobility impairment.

    With regard to JavaScript, the W3C Web Content Accessibility Guidelines 2 (WCAG 2) are more relaxed about JavaScript than WCAG 1 was. Virtually every browser out there today supports JavaScript, and client-side scripting is a necessary piece of today’s modern web infrastructure: remember that most Adaptive Technology interacts with these web browsers, so the pitfalls and problems of relying on JavaScript for ‘mission critical’ functionality are equally present for all users. Again, the real answer to this problem is education.

    5) Let’s talk assistive technology. As markup and scripting should not only be targeted towards certain browsers we need to know what for example screen readers can do these days. How is the support for the new technologies?

    First, I define Assistive Technology as a collection of off-the-shelf software tools, specialized programs, and alternate combinations of hardware; all are Assistive Technology: Dragon not only types when you talk, but provides a hands-free solution to UI interaction to those with severe mobility issues; screen readers are programs that communicate between browsers and other UI tools, and the operating system, leveraging the OS’s accessibility APIs; and Braille output bars connected to various devices allowing for a specialize output for those that require it. OS level offerings such as VoiceOver has been a boon to non-sighted users (allowing them to interact with their iPhones and iPads) and can also be considered AT.

    Screen readers are sophisticated software, and while some screen readers such as NVDA are actively working to keep up with improvements in HTML5, the majority are sadly (I believe) waiting for a more formal standard to settle down before committing resources to re-factor their software. Whether this is the right decision or not is not really the question – it isn’t a question, it is an apparent fact.

    It is a difficult chicken and egg scenario, and it is one of the reasons why I continue to advise that production class development proceed cautiously when using many of the new “HTML5″ technologies. The silver lining in this story is the fact that ARIA has pretty good support in the current screen readers, and so developers can safely rely upon most of ARIA today, in concert with many of the HTML5 goodies. Many of the current UI JavaScript libraries (such as YUI3 and jQuery UI) are also helping out here, so working devs should be looking at a robust blend of current and new technology solutions.

    6) One very important thing to make a document understandable for assistive technology is maintaining a logical order, especially when it comes to headings. With HTML5, we have sections and articles, hgroups and heading order in parts of the document rather than the full document. There is even implicit outlining of the document. How does assistive technology deal with that? Do we block out users by doing the right thing?

    This is actually a very topical issue right now, as recently (January2011) many have come to question the viability and usefulness of the hgroup element in HTML5. W3C Process is such that any real dialog and discussion on this issue is being deferred to later this spring, as the Working Group is trying to clear older bugs and issues that were raised prior to the end of September 2010 cut-off date. I believe that currently no browser supports hgroup

    The outline algorithm for headings is a great idea which has not yet seen implementation or support in Adaptive Technology. The idea is that the browser would heuristically know that an <hx> element was actually supposed to be at an appropriate level (say <h3>) based upon its surrounding ‘inheritance’.

    This makes the outline and structure of a composite document more legible and understandable from the DOM on out to user-interfaces, whether the GUI of the browser, or Adaptive Technology. In effect the browsers would ‘re-write’ the heading level. The need for doing something like this has been understood and around for some time: XHTML2 had proposed a numberless <h> element that would have helped achieve the same goal, but for whatever reasons (likely the simple aversion to XHTML2 as being “wrong”) the idea was not carried into the work of the HTML5 group. Despite the lack of support for this feature today however, the appropriate usage of headings to ‘chunked’ content is still an important requirement for users (and especially screenreader users) – even if the actual heading order in the DOM today is ‘incorrect’ screen readers can and do use headings to facilitate in-page navigation. Today, it’s just not pretty (and sometimes – often – incorrectly ‘stacked’), but useful still the same.

    Sections and Articles are 2 elements that have not yet really seen any kind of actual implementation in the browsers (although I believe that Firefox 4 will start to support them ‘natively’), so at this time they are still in the “this is conceptually a good idea” stage for all practical purposes. However, for browsers that do not have native support for these elements, they treat them as non-semantic divs, and so by adding ARIA landmark roles today they can be made quite navigable/accessible to AT. As a forward looking best practice I think that it is one of the things that developers can start implementing today – just don’t forget the ARIA landmark roles.

    7) One thing I really loved when I found it in the HTML5 specs is a definition of figures and captions. However, when you use them and you want support for assistive technology you need to use ARIA labeled-by and assign a unique ID for every figure to connect it logically with the caption. That is extra effort a lot of developers will not go through. I find a lot of ARIA things very verbose. Are there any efforts to streamline the communication between the different standard bodies?

    Absolutely!

    ARIA as a specification/technology has been with us for a long time – almost 10 years now – but it has only been within the past 3 to 5 years that we’ve seen any real support in tools such as screen readers. ARIA however was built as a bridging technology, and one that was designed as much for remediation as forward development: go back and add the following bits of stuff to make your existing widget accessible. Because of this, yes, sadly, it is hardly elegant or compact. It has been however a chicken and egg problem since it was first addressed.

    One of the significant pieces of work that the W3C Accessibility TaskForce has focused on has been ensuring that ARIA functionality is being integrated and mapped back to emergent HTML5 elements and attributes. While this work is not yet completed, there is a commitment to ensure that it is completed before HTML5 (the markup language) becomes a full recommendation.

    Looking at the 2 examples you pointed out – figures and captions – the longer range plan is that these will map directly to the browser parsers and on to the Accessibility APIs, so that down the road they will simply “work” as advertised on the tin.

    Adding ARIA attributes today is a belt and suspenders approach to both future-proof your content, but also ensure that it is accessible today. In many ways, it is very similar to the vendor prefixes used in CSS3: the ideal is to simply declare the property, but because it is still emergent technology, today we need to prepend the property with a vendor prefix (as in, for example: -moz-margin-end, -webkit-margin-end – where you must declare it twice in your style sheet to target both browsers). Same basic idea: yes, more work for developers, but there is no easy answer.

    8) What do you consider the biggest problem of HTML5 video and audio to date? Is it the lack of options to protect premium content? Or the lack of a supported format for subtitling and captioning? Encoding woes?

    Actually, I think all of the above contribute to a lack of maturity today.

    The encoding issue will remain (I believe) a complicated political dance for some time into the future, with the average content producer left with no choice but to encode twice if they want to use <video> natively for all browsers. HTML5 currently lacks a means to fully support the identified needs of users with various need, although the proposed <track> element is certainly going to help here (but AFAIK no browser is supporting it today,or the ability to extract <track> data via a native interface).

    I fear that the time-stamp format is a debate not yet completed (despite WHAT WG’s overwhelming support for WebSRT, err WebVTT, or whatever they decide to call it next). I think this was all but guaranteed when the SMTPE (Society of Motion Picture and Television Engineers) announced their industry supported SMPTE Time Text Format, based upon TTML. When commercial content providers make up their own minds about a time format,one would suspect that the browser vendors (pressured by online content distributors) will pay attention (especially given the current mood towards web captioning in Washington). While no browser today has staked a firm either/or position here, I get a sense that at least Microsoft will likely support both.

    Issues surrounding DRM and parental controls – admittedly unpopular with a generation used to Pirate Bay and torrent feeds – are both issues that should be addressed,but likely will not be any time soon, further retarding widespread implementation and uptake in my opinion: commercial content providers will want *some* form of DRM, even if it can be hacked – the hacking becoming as much the criminal act as taking the digital asset.

    Meanwhile, alternative solutions to the native <video> solution, from Flash and Silverlight to QuickTime itself, are quietly developing and shipping DRM solutions to clients eager to have this form of protection for their content – just look at Netflix. What I find curious is that many of those loud voices decrying DRM have no issue buying MP3s from iTunes (which can only be shared on 3 machines registered with Apple).

    9) Canvas seems to be another big accessibility issue. Do you know of anything that is being done to make the changes you plot onto a canvas element understandable to assistive technology? Does SVG suffer from the same drawbacks?

    Yes, Canvas remains a tricky accessibility challenge. Sadly, I’ve not been as focused on that aspect as I have on the media stuff, but I have faith and trust in the accessibility folk who are working out the kinks. RichardS chwerdtfeger (IBM) is the chair of the Canvas Accessibility sub-team and is a very smart engineer, and all of the browser vendors are in active discussion in that sub-team. My understanding is that there is a general consensus on the subtree DOM, although cross browser support for the solution is not yet there. There remains issues with caret focus and textMetrics interaction with screen magnifiers, as well as how to convey absolute positioning of all content to screen magnifiers and screenreaders who make use of Braille.

    SVG, as a technology, is very different than Canvas and for the most part there are few accessibility issues with SVG. There’s a good over-view of accessibility Features of SVG available.

    10) The silver bullet for safely using new technologies seems to be object detection. You test if the browser supports something before you apply it. This never worked with assistive technology. Why is that the case? Why can’t we just say if(window.screenreader)?

    There are a number of reasons why this is not feasible.

    For one, “screenreader” is a class of software tools that all behave slightly differently. Yes, today in the English speaking “Western” world,the JAWS software package holds a commanding share, but alternatives such as WindowEyes, Hal/Supernova, ZoomText and upstarts like NVDA and Serotek’s SAToGo are challenging the status quo. Yet none of these tools behave in exactly the same way, and at times their approach to user-interaction can differ greatly. Add foreign language tools such as the Korean Sense Reader Professional and the Brazilian Leitor de Telas – to name but two – and sniffing for a screen reader becomes somewhat futile: if you know I am using SAToGo (for example), what does that mean to you as a developer, exactly?

    Things get worse however, as not only do you need to account for ‘brand’ of screenreader, but also version. While WebAIM’s Screen Reader Survey last year confirmed that most users keep their software up-to-date (74.6% within the first year), that still leaves almost 1 in 4 using software that can be as much as 3 years old or older. Developers need to remember as well that screenreaders are not part of the browser, but rather an OS level software tool that interacts not only with the browser, but also with other tools such as text editors, spread-sheet applications, and other productivity tools on their computers.

    Finally what of VoiceOver? The OS level screen reading option from Apple is present on virtually every system they ship today, from OSx to iOS – there is no guarantee that simply because it is there, it is being used – or that the person using it is in fact blind. In fact many sighted users will have a legitimate need or desire to use screen reading technology, as VoiceOver has confirmed, so custom tailoring an ‘experience’ targeted to screen readers is sadly an effort not worth pursuing – you will continue to miss as often as you hit. (It kind of reminds me of web sites that will force an alternative version of their site targeted and optimized to mobile devices, and provide no means of getting to the actual desktop version of the site, even if the user wants that).

    What I tell developers is this: code to standards, think about graceful degradation as well as progressive enhancement, and remember the original three legged stool approach to web development: the separation of content, design and scripting. I remind them that people with disabilities have a social reasonability to keep their software relatively up-to-date as well (no more or less so than any other user – who supports Netscape 4 today?) and/but as a designer/developer you must always be considering the “PlanB” for user interaction – if “Plan A” fails, what is “Plan B”?

    Photo by Dirk Ginader

    Do you know anyone I should interview for “People of HTML5″? Tell me on Twitter: @codepo8

  9. People of HTML5 – Rob Hawkes

    HTML5 needs spokespeople to work. There are a lot of people out there who took on this role, and here at Mozilla we thought it is a good idea to introduce some of them to you with a series of interviews and short videos. The format is simple – we send the experts 10 questions to answer and then do a quick video interview to let them introduce themselves and ask for more detail on some of their answers.

    Leggi la traduzione in italiano

    Rob HawkesToday we are featuring Rob Hawkes author of Rawkets, a multiplayer game using canvas and websockets.

    Rob was one of the presenters at the Game On 2010 event in London, England where he showed off his game. The video and his slides are available on the Mozilla Games Blog. He is currently busy working on a book about Canvas. You can find Rob on Twitter as @robhawkes.

    The message I really loved getting from Rob was that it is OK to use any technology you want and that there is a benefit to people messing around with your code – even if it is to game the system.

    The video interview

    You can see the video on any HTML5 enabled device here (courtesy of vid.ly).

    Ten questions about HTML5 for Rob Hawkes

    1) So, you’ve been fiddling around with Canvas for gaming a lot – why the fascination? Did you write games in other environments before?

    I sure have. My life has practically revolved around making games of some sort for the past year or two now. I just love the concept of using games to experiment with a programming languages and really learn everything I can about them. It’s a good way to try out most of the features involved in visual programming as well, particularly with harder concepts like physics, which I’m still no expert in. My fascination with games programming started out of necessity really; I had to create a game in my second year of university, so I co-created an augmented reality game using Papervision3D and ActionScript. From there I co-created another game of sorts in Adobe Flex that allowed people to play against each other using a webcam, which was fun. And now I’m making games in canvas; partly because it’s fun, and partly to learn everything I can about it. I get a kick out of the interactivity and playability factor that games bring, which I think hooks into my addiction for visual programming in general.

    2) The cool thing about open technologies is that it is easy to see what is going on under the hood. View source and you get the picture. For gaming, isn’t that a problem? To “game” a highscore for a Flash game at least you need to do some HTTP sniffing – with Canvas it must be even easier. How do you protect yourself from that?

    The fact that canvas is so open is something that makes me interested in it. I’m usually pretty open about programming and I’d love it if people can learn from my code, so being able to view the source code directly within the browser is amazing for that. However, the major drawback is that your code is open for everyone to see, so security becomes a pretty big issue.

    You’re right though, this is particularly troublesome with gaming, and I’m inclined to say that there isn’t much that you can do about it. In fact, is wasn’t long after I released the first version of Rawkets that people started to poke around the code and manipulate the game to their advantage.

    In all honesty, I actually see it as a good thing, as it showed me where my code was weak, and highlighted the areas that I needed to secure. You can obfuscate and minify your code as much as you want, but people can still get access to it because it’s just the way JavaScript works. However, there are a few ways to mitigate the problem, like passing core logic for the game to a remote server and communicating to it via restrictive and secure API calls. This is a good method for multiplayer games, where data received from the server is seen as trustworthy, and where as data on the client side is not. Or, you can wrap sensitive code in closures and take advantage of local variables to hide away data that you don’t want people to easily access. Another option is to just not worry about it. So long as you don’t let the hackers ruin the game for other players, does it really matter if they screw it up in their own browser?

    In reality, none of these methods will 100% protect you, but they’ll certainly make things harder for would-be hackers.

    3) Talking about open. Isn’t Canvas just a Java applet with a predefined API and no need for a slow JVM? You still only have a rectangle somewhere in the page you can manipulate…

    If you mean that canvas is like a sandbox that you can only access via a specific set of API calls, then yes, pretty much. Then again, the API calls allow you to do a hell of a lot, and couldn’t you say the same about Flash and all the closed methods of creating graphics?

    I suppose when you say canvas is open; you’re mainly talking about the development behind its creation (ie. the W3C), and the fact that the code to draw elements on it isn’t hidden away from prying eyes. Canvas is definitely not open in the sense that you are free to extend it and do whatever you want with it. So long as you abide by the API calls, which is fine for 99% of the time, then you’ve not got a problem.

    As you say, canvas is effectively just a rectangular area that the browser allows you to draw into. Once you’ve drawn a few shapes into it, you can’t access them individually, but then again, that’s exactly how canvas is meant to work. It’s not like SVG, where every element that you’ve drawn can be accessed from the DOM and edited. Canvas is a bitmap system that has no concept of individual elements. It’s a destructive system that basically just changes the colour of pixels within its rectangle, based on the API calls that you give it. In that way, it’s just like Microsoft Paint from back in the day.

    4) What about accessibility? Can assistive technology access canvas content? How about keyboard access?

    To put it bluntly, not really. At least, not yet.

    Canvas does have fallback content, which is placed between the canvas tags and shows up when a browser doesn’t support canvas, but it doesn’t have much else to aid with accessibility. The cool thing about the fallback content is that it’s possible to place content in there for keyboard access. I believe IE9 already has this functionality, which lets you show the canvas, but still give keyboard access so you can describe what’s going in within the canvas in text, or something like that. The problem with methods like this is that you effectively duplicate all of your content; you draw everything on the canvas, and then you have to write it all up within the canvas tags to explain what you’ve drawn.

    What you can’t do yet is give assistive access to individual elements within the canvas, simply because canvas is just a big rectangle of pixels with no concept about what’s bee drawn. This is the major drawback with using a bitmap system that has a pretty closed API. There is talk about using something called the shadow DOM, to open up the canvas and allow some sort of access into the elements that have been drawn, but I don’t believe much has been done about this yet.

    It’s definitely an area that I’m interested to see progress, but in the meantime, Bruce Lawson highlights a couple of the proposals that are floating around to sort out the accessibility issue with canvas.

    5) What about performance? What are the hot tips to keep a canvas game run smoothly?

    This is a massive sticking point with canvas at the moment; it can be so god damn slow! Actually, this isn’t really a problem with canvas, but rather the fact that canvas uses the browser process to draw everything, which in turn uses the processor (CPU).

    Some browsers are introducing something called hardware acceleration, which passes off canvas and graphical processes to the graphics card (GPU). This simple feature really speeds things up, and allows you to do some pretty intensive stuff. Still, hardware acceleration isn’t a magic bullet; you still need to program defensively and be aware of potential bottlenecks and performance issues with your code.

    The most common issues arise around unnecessarily extravagant timers and loops. For example, animating in canvas usually requires the use of a JavaScript timeout, which runs a function that draws everything onto the canvas 30 times a second; fast enough to look like things are moving. This is great, but what about if you’re not animating things any more? I’ve made the mistake of not turning off the animation timer, and inherently sucking precious CPU juice for no good reason.

    Another example is that of collision detection, or any other check that has to be performed on a large amount of objects. The bad way of doing this is to loop through every object, and then loop through every object a second time on each iteration of the original loop. The problem with this is that you end up checking some objects twice, or more, and basically wasting resources. Instead, you can optimise your loops to ignore objects that you’ve already checked, which can cut the amount of loops down considerably.

    There are countless ways to optimise your code, and most won’t achieve much on their own, but together they can add up to save a pretty significant amount of resources.

    6) Your game uses Web sockets for communication. How did you find that to work with? Does it scale well? Are there any things you don’t like about the technology?

    You mean that it used to use WebSockets? It’s a shame FireFox and others decided it would be a great idea to drop support for them for the time being. Jesting aside, the game is built atop WebSockets, and it’s been an interesting journey. I love WebSockets, and I think that having a full-duplex method of communicating within a browser is fantastic. The fact that the data is streamed live opens up a whole range of possibilities for games and Web apps.

    I didn’t find too many issues with WebSockets regarding stability and scalability, as in reality most of the problems lay with the code on the server that deals with sending information back and forth. I found that one thing you need to be aware of is that the amount of data being sent back and forth can increase at an exponential rate if you’re not used to optimising communication for a large amount of users.

    I have to admit though, my biggest bug bear with WebSockets is that it sends data as text, and it doesn’t support the UDP protocol. Right now, WebSockets sending everything as text data is great because it’s dead simple, but it sucks because it can potentially take a lot of precious bandwidth, especially when you’re sending data many times a second to hundreds of different players in a game.

    The UDP problems relates to how canvas currently only supports the TCP protocol. What this means is that every piece of data that is sent back and forth between a server and a client has to be received in order. This is not good for gaming, where a bit of lag can end up causing a backlog of messages that have to be sent in order before any new messages can be sent. UDP on the other hand would allow you to send messages as and when they occur, without worrying about order. This is very good for games that rely on precise data that is time-sensitive, like the movement of a player in an FPS game.

    You have to remember that WebSockets is something new, and it’s likely to have some teething troubles at the beginning; hence it being disabled temporarily. Still, it’s got a bright future, and I’m looking forward to seeing how it progresses.

    7) Have you played with SVG, too? If so, when would you use what and is there a chance to use the strong parts of both technologies together?

    Admittedly, I’ve not played with SVG as much as I’d like to. I’ve just become so caught up in the world of canvas in recent months. However, it’s something that I strongly believe should be used instead of canvas in particular circumstances, like when dealing with the conversion of existing HTML data into a graphical format (eg. creating graphs from a table).

    SVG is also great because it’s accessible via the DOM, but it falls down with animation and gaming. It was never built with this kind of use in mind, and it’s exactly why canvas exists. Canvas is perfect for fast and dynamic animations, and it’s particularly perfect for creating games.

    Fortunately, it isn’t always a case of being either or with SVG and canvas. It’s completely possible to draw SVG shapes within canvas, which has the added benefit of letting you have all the vector goodness of SVG within a bitmap system. I’ve heard of people using techniques like this for sprites within games, and for drawing images onto canvas that need to be resized a lot.

    8) What’s the next barrier you’d love to see opened up? What features of closed technology are there that are beyond your reach but you are itching to try out?

    I’m really interested in seeing the Device APIs (DAP) being supported in the major browsers. One of the things that is lacking in browsers at the moment is support for external devices like webcams and microphones. I can just see canvas and DAP being used together for some pretty awesome Web apps and visualisations. Imagine being able to create an avatar for a website by using a bit of canvas with an attached webcam. Or, what about live video chat through the browser with WebSockets and canvas/HTML5 video? That would be JavaScript awesomeness right there!

    9) Currently you’re writing a book on Canvas. As not everybody has time to wait for this, where could people go in the meantime to get the real info about starting with canvas?

    It’s not a massive wait (it’ll be out in May), but in the meantime I’d suggest going to the Mozilla Developer Network. It’s where I learnt how to use canvas, so it must be good! Apart from that, there really is a lack of decent resources for learning canvas at the moment, but I predict that changing for the better over the next year.

    10) Where could people go to get inspiration? What other uses of Canvas have you seen that made you go “woooo I want that!”?

    There are some fantastic projects being created with canvas at the moment. I could create a massive list, but here are some of my favourite:

    Some particularly impressive uses of canvas that have got my juices flowing are projects that really push the pixel manipulation features of canvas, like using it to detect faces in images, or nudity. Weird, but pretty impressive stuff!

    Do you know anyone I should interview for “People of HTML5″? Tell me on Twitter: @codepo8

  10. People of HTML5 – Remy Sharp

    HTML5 needs spokespeople to work. There are a lot of people out there who took on this role, and here at Mozilla we thought it is a good idea to introduce some of them to you with a series of interviews and short videos. The format is simple – we send the experts 10 questions to answer and then do a quick video interview to let them introduce themselves and ask for more detail on some of their answers.

    Leggi la traduzione in italiano

    Remy SharpToday we are featuring Remy Sharp co-author of Introducing HTML5 and organiser of the Full Frontal conference in Brighton, England.

    Remy is one of those ubiquitous people of HTML5. Whenever something needed fixing, there is probably something on GitHub that Remy wrote that helps you. He is also very English and doesn’t mince his words much.

    You can find Remy on Twitter as @rem.

    The video interview

    Watch the video on YouTube or Download it from Archive.org as MP4 (98 MB), OGG (70 MB) or WebM (68MB)

    Ten questions about HTML5 for Remy Sharp

    1) Reading “Introducing HTML5″ it seems to me that you were more of the API – focused person and Bruce the markup guy. Is that a fair assumption? What is your background and passion?

    That’s spot on. Bruce asked me to join the project as the “JavaScript guy” – which is the slogan I wear under my clothes and frequently reveal in a superman ‘spinning around’ fashion (often at the surprise of clients).

    My background has always been coding – even from a young age, my dad had me copying out listings from old spectrum magazines only to result in hours of typing and some random error that I could never debug.

    As I got older I graduated to coding in C but those were the days the SDKs were 10Mb downloaded over a 14kb modem, and compile in to some really odd environment. Suffice to say I didn’t get very far.

    Then along came JavaScript. A programming language that didn’t require any special development environment. I could write the code in Notepad on my dodgy Window 95 box, and every machine came with the runtime: the browser. Score!

    From that point on the idea of instant gratification from the browser meant that I was converted – JavaScript was the way for me.

    Since then I’ve worked on backend environments too (yep, I’m a Perl guy, sorry!), but always worked and played in the front end in some way or another. However, since started on my own in 2006, it’s allowed me to move focus almost entirely on the front end, and specialise in JavaScript. Basically, work-wise: I’m a pig in shit [Ed: for our non-native English readers, he means "happy")].

    2) From a programmer’s point of view, what are the most exciting bits about the HTML5 standard? What would you say is something every aspiring developer should get their head around first?

    For me, the most exciting aspects of HTML5 is the depth of the JavaScript APIs. It’s pretty tricky to explain to Joe Bloggs that actually this newly spec’ed version of HTML isn’t mostly HTML; it’s mostly JavaScript.

    I couldn’t put my finger on one single part of the spec, only because it’s like saying which is your favourite part of CSS (the :target selector – okay, so I can, but that’s not the point!). What’s most exciting to me is that HTML5 is saying that the browser is the platform that we can deliver real applications – take this technology seriously.

    If an aspiring developer wanted something quick and impressive, I’d say play around with the video API – by no means is this the best API, just an easy one.

    If they really wanted to blow people away with something amazing using HTML5, I’d say learn JavaScript (I’m assuming they’re already happy with HTML and CSS). Get a book like JavaScript: The Good Parts and then get JavaScript Patterns and master the language. Maybe, just maybe, then go buy Introducing HTML5, it’s written by two /really/ good looking (naked) guys: http://flic.kr/p/8iyQTE and http://flic.kr/p/8iy6Z1 [Ed: maybe NSFW, definitely disturbing].

    3) In your book you wrote a nice step-by-step video player for HTML5 video. What do you think works well with the Video APIs and what are still problems that need solving?

    The media API is dirt simple, so it means working with video and audio is a doddle. For me, most of it works really well (so long as you understand the loading process and the events).

    Otherwise what’s really quite neat, is the fact I can capture the video frames and mess with them in a canvas element – there’s lots of fun that can be had there (see some of Paul Rouget’s demos for that!).

    What sucks, and sucks hard, is the spec asks vendors, ie. browser makers, *not* to implement full screen mode. It uses security concerns as the reason (which I can understand), but Flash solved this long ago – so why not follow their lead on this particular problem? If native video won’t go full screen, it will never be a competitive alternative to Flash for video.

    That all said, I do like that the folks behind WebKit went and ignored the spec, and implemented full screen. The specs are just guidelines, and personally, I think browsers should be adding this feature.

    4) Let’s talk a bit about non-HTML5 standards, like Geolocation. I understand you did some work with that and found that some parts of the spec work well whilst others less so. Can you give us some insight?

    On top of HTML5 specification there’s a bunch more specs that make the browser really, really exciting. If we focus on the browser being released today (IE9 included) there’s a massive amount that can be done that we couldn’t do 10 years ago.

    There’s the “non-HTML5″ specs that actually were part of HTML5, but split out for good reason (so they can be better managed), like web storage, 2D canvas API and Web Sockets, but there’s also the /really/ “nothing-to-do-with-HTML5″ APIs (NTDWH5API!) like querySelector, XHR2 and the Device APIs. I’m super keen to try all of these out even if they’re not fully there in all the browsers.

    Geolocation is a great example of cherry picking technology. Playing against the idea that the technology isn’t fully implemented. Something I find myself ranting on and on about when it comes to the question of whether a developer should use HTML5. Only 50% of Geolocation is implemented in the browsers supporting it, in that they don’t have altitude, heading or speed – all of which are part of the spec. Does that stop mainstream apps like Google Maps from using the API? (clue: no).

    The guys writting the specs have done a pretty amazing job, and in particular there are few cases where the specs have been retrospectively written. XHR is one of these and now we’ve got a stable API being added in new browsers (i.e. IE6 sucks, yes, we all know that). Which leads us to drag and drop. The black sheep of the APIs. In theory a really powerful API that could make our applications rip, but the technical implementation is a shambles. PPK (Peter-Paul Koch) tore the spec a bit of a ‘new one’. It’s usable, but it’s confusing, and lacking.

    Generally, I’ve found the “non-HTML5″ specs to be a bit of mixed bag. Some are well supported in new browsers, some not at all. SVG is an oldie and now really viable with the help of JavaScript libraries such as Raphaël.js or SVGWeb (a Flash based solution). All in all, there’s lots of options available in JavaScript API nowadays compared to back in the dark ages.

    5) Let’s talk Canvas vs. SVG for a bit. Isn’t Canvas just having another pixel-based rectangle in the page much like Java Applets used to be? SVG, on the other hand is Vector based and thus would be a more natural tool to do something with open standards that we do in Flash now. When would you pick SVG instead of Canvas and vice versa?

    Canvas, in a lot of ways is just like the Flash drawing APIs. It’s not accessible and a total black box. The thing is, in the West, there’s a lot of businesses, rightly or wrongly, that want their fancy animation thingy to work on iOS. Since Flash doesn’t work there, canvas is a superb solution.

    However, you must, MUST, decide which technology to use. Don’t just use canvas because you saw a Mario Kart demo using it. Look at the pros and cons of each. SVG and the canvas API are not competitive technologies, they’re specially shaped hammers for specific jobs.

    Brad Neuberg did a superb job of summarising the pros and cons of each, and I’m constantly referring people to it (here’s the video).

    So it really boils down to:

    • Canvas: pixel manipulation, non-interactive and high animation
    • SVG: interactive, vector based

    Choose wisely young padawan!

    6) What about performance? Aren’t large Canvas solutions very slow, especially on mobile devices? Isn’t that a problem for gaming? What can be done to work around that?

    Well…yes and no. I’m finishing a project that has a large canvas animation going on, and it’s not slow on mobile devices (not that it was designed for those). The reason it’s not slow is because of the way the canvas animates. It doesn’t need to be constantly updating at 60fps.

    Performance depends on your application. Evaluate the environment, the technologies and make a good decision. I personally don’t think using a canvas for a really high performance game on a mobile is quite ready. I don’t think the devices have the ommph to get the job done – but there’s a hidden problem – the browser in the device isn’t quite up to it. Hardware acceleration is going to help, a lot, but today, right now, I don’t think we’ll see games like Angry Birds written in JavaScript.

    That said… I’ve seriously considered how you could replicate a game like Canabalt using a mix of canvas, DIVs and CSS. I think it might be doable ::throws gauntlet::

    I think our community could actually learn a lot from the Flash community. They’ve been through all of this already. Trying to make old versions of Flash from years back do things that were pushing the boundaries. People like Seb Lee-Delisle (@seb_ly / http://seb.ly) are doing an amazing job of teaching both the Flash and JavaScript community.

    7) A feature that used to be HTML5 and is now an own spec is LocalStorage and its derivatives Session Storage or the full-fledged WebSQL and IndexedDB. Another thing is offline storage. There seems to be a huge discussion in developer circles about what to use when and if NoSQL solutions client side are the future or not. What are your thoughts? When can you use what and what are the drawbacks?

    Personally I love seeing server-less applications. Looking at the storage solutions I often find it difficult to see why you wouldn’t use WebStorage every time.

    In a way it acts like (in my limited experience of) NoSQL, in that you lookup a key and get a result.

    Equally, I think SQL in the browser is over the top. Like you’re trying to use the storage method *you* understand and forcing it into the browser. Seems like too much work for too little win.

    Offline Apps, API-wise, ie. the application cache is /really/ sexy. Like sexy with chocolate on top sexy. The idea that our applications can run without the web, or switch when it detects it’s offline is really powerful. The only problem is that the events are screwed. The event to say your app is now offline requires the user to intervene via the browser menu, telling the browser to “work in offline mode”. A total failure of experience. What’s worse is that, as far as I know, there’s no plan to make offline event fire properly :-(

    That all said, cookies are definitely dead for me. I’ve yet to find a real solution for cookies since I found the Web Storage API – and there’s a good decent number of polyfills for Web Storage – so there’s really no fear of using the API.

    8) Changing the track a bit, you’ve built the HTML5shiv to make HTML5elements styleable in IE. This idea sparked quite a lot of other solutions to make IE6 work with the new technologies (or actually simulate them). Where does this end? Do you think it is worth while to write much more code just to have full IE6 support?

    There’s two things here:

    1. Supporting IE6 (clue: don’t)
    2. Polyfills

    IE6, seriously, and for the umpteenth time, look at your users. Seriously. I know the project manager is going to say they don’t know what the figures are, in that case: find out! Then, once you’ve got the usage stats in hand, you know your audience and you know what technology they support.

    If they’re mostly IE6 users, then adding native video with spinning and dancing canvas effect isn’t going to work – not even with shims and polyfills. IE6 is an old dog that just isn’t up to doing the mileage he used to be able to do back in his prime. But enough on this subject – the old ‘do I, or don’t I developer for IE6′ is long in the tooth.

    Polyfills – that’s a different matter. They’re not there to support IE6, they’re there to bring browsers up to your expectations as a developer. However, I’d ask that you carefully consider them before pulling them in. The point of these scripts is they plug missing APIs in those older browsers. “Old browsers” doesn’t particularly mean IE6. For example, the Web Sockets API has a polyfill by way of Flash. If native Web Sockets aren’t there, Flash fills the gap, but the API is exposed in exactly the same manner, meaning that you don’t have to fork your code.

    I don’t think people should be pulling in scripts just for the hell of it. You should consider what you’re trying to achieve and decide whether X technology is the right fit. If it is, and you know (or expect) your users have browsers that don’t support X technology – should you plug it with JavaScript or perhaps should you consider a different technology?

    This exact same argument rings true for when someone adds jQuery just to add or remove a class from an element. It’s simply not worth it – but clearly that particular developer didn’t really understand what they needed to do. So is education the solution? I should hope so.

    9) Where would you send people if they want to learn about HTML5? What are tutorials that taught you a lot? Where should interested people hang out?

    HTML5 Doctor – fo sho’. :)

    I tend to also direct people to my http://html5demos.com simply to encourage viewing source, and hacking away.

    Regarding what tutorials taught me – if I’m totally honest, the place I’ve learnt the most from is actually HTML5Doctor.com. There’s some pretty good JavaScript / API tutorials coming from the chaps at http://bocoup.com. Otherwise, I actually spend a lot of time just snooping through the specifications, looking for bits that I’ve not seen before and generally poking them with a stick.

    10) You have announced that you are concentrating on building a framework to make Websockets easy to work with. How is that getting on and what do you see Websockets being used for in the future? In other words, why the fascination?

    Concentrating is a strong word ;-) but it is true, I’ve started working on a product that abstracts Web Sockets to a service. Not the API alone, since it’s so damn simple, but the server setup: creating sessions, user control flow, waiting for users and more.

    The service is called Förbind. Swedish for “connect”, ie. to connect your users. It’s still early days, but I hope to release alpha access to forbind.net this month.

    I used to work in finance web sites and real-time was the golden egg: to get that data as soon as it was published. So now that it’s available in a native form in the browser, I’m all over it!

    What’s more, I love the idea of anonymous users. I created a bunch of demos where the user can contribute to something without ever really revealing themselves, and when the users come, you start to see how creative people are without really trying. Sure, you get a lot of cocks being drawn, but you also see some impressive ideas – my business 404 page for example allows people to leave a drawing, one of the most impressive is a Super Mario in all his glory. Anonymous users really interest me because as grey as things can seem sometimes, a stranger can easily inspire you.

    Do you know anyone I should interview for “People of HTML5″? Tell me on Twitter: @codepo8