Mozilla

Audio Articles

Sort by:

View:

  1. It’s Opus, it rocks and now it’s an audio codec standard!

    In a great victory for open standards, the Internet Engineering Task Force (IETF) has just standardized Opus as RFC 6716.

    Opus is the first state of the art, free audio codec to be standardized. We think this will help us achieve wider adoption than prior royalty-free codecs like Speex and Vorbis. This spells the beginning of the end for proprietary formats, and we are now working on doing the same thing for video.

    There was both skepticism and outright opposition to this work when it was first proposed in the IETF over 3 years ago. However, the results have shown that we can create a better codec through collaboration, rather than competition between patented technologies. Open standards benefit both open source organizations and proprietary companies, and we have been successful working together to create one. Opus is the result of a collaboration between many organizations, including the IETF, Mozilla, Microsoft (through Skype), Xiph.Org, Octasic, Broadcom, and Google.

    A highly flexible codec

    Unlike previous audio codecs, which have typically focused on a narrow set of applications (either voice or music, in a narrow range of bitrates, for either real-time or storage applications), Opus is highly flexible. It can adaptively switch among:

    • Bitrates from 6 kb/s to 512 kb/s
    • Voice and music
    • Mono and stereo
    • Narrowband (8 kHz) to Fullband (48 kHz)
    • Frame sizes from 2.5 ms to 60 ms

    Most importantly, it can adapt seamlessly within these operating points. Doing all of this with proprietary codecs would require at least six different codecs. Opus replaces all of them, with better quality.
    Illustration of the quality of different codecs
    The specification is available in RFC 6716, which includes the reference implementation. Up-to-date software releases are also available.

    Some audio standards define a normative encoder, which cannot be improved after it is standardized. Others allow for flexibility in the encoder, but release an intentionally hobbled reference implementation to force you to license their proprietary encoders. For Opus, we chose to allow flexibility for future encoders, but we also made the best one we knew how and released that as the reference implementation, so everyone could use it. We will continue to improve it, and keep releasing those improvements as open source.

    Use cases

    Opus is primarily designed for use in interactive applications on the Internet, including voice over IP (VoIP), teleconferencing, in-game chatting, and even live, distributed music performances. The IETF recently decided with “strong consensus” to adopt Opus as a mandatory-to-implement (MTI) codec for WebRTC, an upcoming standard for real-time communication on the web. Despite the focus on low latency, Opus also excels at streaming and storage applications, beating existing high-delay codecs like Vorbis and HE-AAC. It’s great for internet radio, adaptive streaming, game sound effects, and much more.

    Although Opus is just out, it is already supported in many applications, such as Firefox, GStreamer, FFMpeg, foobar2000, K-Lite Codec Pack, and lavfilters, with upcoming support in VLC, rockbox and Mumble.

    For more information, visit the Opus website.

  2. getUserMedia is ready to roll!

    We blogged about some of our WebRTC efforts back in April. Today we have an exciting update for you on that front: getUserMedia has landed on mozilla-central! This means you will be able to use the API on the latest Nightly versions of Firefox, and it will eventually make its way to a release build.

    getUserMedia is a DOM API that allows web pages to obtain video and audio input, for instance, from a webcam or microphone. We hope this will open the possibility of building a whole new class of web pages and applications. This DOM API is one component of the WebRTC project, which also includes APIs for peer-to-peer communication channels that will enable exchange of video steams, audio streams and arbitrary data.

    We’re still working on the PeerConnection API, but getUserMedia is a great first step in the progression towards full WebRTC support in Firefox! We’ve certainly come a long way since the first image from a webcam appeared on a web page via a DOM API. (Not to mention audio recording support in Jetpack before that.)

    We’ve implemented a prefixed version of the “Media Capture and Streams” standard being developed at the W3C. Not all portions of the specification have been implemented yet; most notably, we do not support the Constraints API (which allows the caller to request certain types of audio and video based on various parameters).

    We have also implemented a Mozilla specific extension to the API: the first argument to mozGetUserMedia is a dictionary that will also accept the property {picture: true} in addition to {video: true} or {audio: true}. The picture API is an experiment to see if there is interest in a dedicated mechanism to obtain a single picture from the user’s camera, without having to set up a video stream. This could be useful in a profile picture upload page, or a photo sharing application, for example.

    Without further ado, let’s start with a simple example! Make sure to create a pref named “media.navigator.enabled” and set it to true via about:config first. We’ve put the pref in place because we haven’t implemented a permissions model or any UI for prompting the user to authorize access to the camera or microphone. This release of the API is aimed at developers, and we’ll enable the pref by default after we have a permission model and UI that we’re happy with.

    There’s also a demo page where you can test the audio, video and picture capabilities of the API. Give it a whirl, and let us know what you think! We’re especially interested in feedback from the web developer community about the API and whether it will meet your use cases. You can leave comments on this post, or on the dev-media mailing list or newsgroup.

    We encourage you to get involved with the project – there’s a lot of information about our ongoing efforts on the project wiki page. Posting on the mailing list with your questions, comments and suggestions is great way to get started. We also hang out on the #media IRC channel, feel free to drop in for an informal chat.

    Happy hacking!

  3. Firefox 4 Beta: Latest Update is Here, with WebGL

    The new Firefox 4 Beta is here, and comes with WebGL activated by default. You can download this new beta here: http://firefox.com/beta.

    Flight of the Navigator is a WebGL + Audio API demo developed by a team of Mozilla volunteers.

    You can see the demo online here (you need a WebGL compatible browser).

    More information about this demo on David Humphrey’s blog (one of the developer of this demo) and more about WebGL in Firefox Beta8 on Vlad’s blog.

    The screencast:

  4. Firefox Beta 15 supports the new Opus audio format

    Firefox 15 (now in the Beta channel) supports the Opus audio format, via the Opus reference implementation.

    What is it?

    Opus is a completely free audio format that was recently approved for publication as a standards-track RFC by the IETF. Opus files can play in Firefox Beta today.

    Opus offers these benefits:

    • Better compression than MP3, Ogg, or AAC formats
    • Good for both music and speech
    • Dynamically adjustable bitrate, audio bandwidth, and coding delay
    • Support for both interactive and pre-recorded applications

    Why Should I care?

    First, Opus is free software, free for everyone, for any purpose. It’s also an IETF standard. Both the encoder and decoder are free, including the fixed-point implementation (for mobile devices). These aren’t toy demos. They’re the best we could make, ready for serious use.

    We think Opus is an incredible new format for web audio. We’re working hard to convince other browsers to adopt it, to break the logjam over a common <audio> format.

    The codec is a collaboration between members of the IETF Internet Wideband Audio Codec working group, including Mozilla, Microsoft, Xiph.Org, Broadcom, Octasic, and others.

    We designed it for high-quality, interactive audio (VoIP, teleconference) and will use it in the upcoming WebRTC standard. Opus is also best-in-class for live streaming and static file playback. In fact, it is the first audio codec to be well-suited for both interactive and non-interactive applications.

    Opus is as good or better than basically all existing lossy audio codecs, when competing against them in their sweet spots, including:

    General audio codecs (high latency, high quality)
    • MP3
    • AAC (all flavors)
    • Vorbis
    Speech codecs (low latency, low quality)
    • G.729
    • AMR-NB
    • AMR-WB (G.722.2)
    • Speex
    • iSAC
    • iLBC
    • G.722.1 (all variants)
    • G.719

    And none of those codecs have the versatility to support all the use cases that Opus does.

    Listening tests show that:

    That’s a lot of bandwidth saved. It’s also much more flexible.

    Opus can stream:

    • narrowband speech at bitrates as low as 6 kbps
    • fullband music at rates of 256 kbps per channel

    At the higher of those rates, it is perceptually lossless. It also scales between these two extremes dynamically, depending on the network bandwidth available.

    Opus compresses speech especially well. Those same test results (slide 19) show that for fullband mono speech, Opus is almost transparent at 32 kbps. For audio books and podcasts, it’s a real win.

    Opus is also great for short files (like game sound effects) and startup latency, because unlike Vorbis, it doesn’t require several kilobytes of codebooks at the start of each file. This makes streaming easier, too, since the server doesn’t have to keep extra data around to send to clients who join mid-stream. Instead, it can send them a tiny, generic header constructed on the fly.

    How do I use it in a web page?

    Opus works with the <audio> element just like any other audio format.

    For example:

     <audio src="ehren-paper_lights-64.opus" controls>

    This code in a web page displays an embedded player like this:


    Paper Lights by Ehren Starks Creative Commons License

     
    (Requires Firefox 15 or later)

    Encoding files

    For now, the best way to create Opus files is to use the opusenc tool. You can get source code, along with Mac and Windows binaries, from:

    http://www.opus-codec.org/downloads/

    While Firefox 15 is the first browser with native Opus support, playback is coming to gstreamer, libavcodec, foobar2000, and other media players.

    Streaming

    Live streaming applications benefit greatly from Opus’s flexibility. You don’t have to decide up front whether you want low bandwidth or high quality, to optimize for voice or music, etc. Streaming servers can adapt the encoding as conditions change—without breaking the stream to the player.

    Pre-encoded files can stream from a normal web server. The popular Icecast streaming media server can relay a single, live Opus stream, generated on the fly, to thousands of connected listeners. Opus is supported by the current development version of Icecast.

    More Information

    To learn more visit opus-codec.org, or join us in #opus on irc.freenode.net.

  5. audio player – HTML5 style

    Last week we featured a demo from Alistair MacDonald (@F1LT3R) where he showed how to animate SVG with Canvas and a bunch of free tools. This week he has another demo for us that shows how you can use the new audio element in Firefox 3.5 with some canvas and JS to build a nice-looking audio player.

    But what’s really interesting about this demo is not so much that it plays audio – lots of people have built audio players – but how it works. If you look at the source code for the page what you’ll find looks something like this:

    <div id="jai">
      <canvas id="jai-transport" width="320" height="20"></canvas>
      <ul class="playlist">
        <li>
          <a href="@F1LT3R - Cryogenic Unrest.ogg">
            F1LT3R - Cryogenic Unrest
          </a>
          <audio src="@F1LT3R - Cryogenic Unrest.ogg"/>.
        <li>
          <a href="@F1LT3R - Ghosts in HyperSpace.ogg">
            F1LT3R - Ghosts in HyperSpace
          </a>
          <audio src="@F1LT3R - Ghosts in HyperSpace.ogg"/>.       
      </ul>    
    </div>
    (The actual list has fallbacks and is more compact – cleaned up here for easier reading.)

    That’s right – the player above is just a simple HTML unordered list that happens to include audio elements and is styled with CSS. You’ll notice that if you right click on one of them that it has all the usual items – save as, bookmark this link, copy this link location, etc. You can even poke at it with Firebug.

    The JavaScript driver that Al has written will look for a <div> element with the jai ID and then look for any audio elements that are inside it. It then will draw the playback interface in the canvas at the top of the list. The playback interface is built with simple JS canvas calls and an SVG-derived font.

    Using this driver it’s super-easy to add an audio player to any web site by just defining a canvas and a list. Much like what we’ve seen on a lot of the web with the rise of useful libraries like jQuery, this library can add additional value to easily-defined markup. Another win for HTML5 and the library model.

    Al has a much larger write-up on the same page as the demo. If you haven’t read through it you should now.

    (Also? Al wrote the music himself. So awesome.)

  6. open video codecs and quality

    This is a re-post (with permission) of a post that Greg Maxwell wrote in response to a comment by Chris DiBona from Google on a whatwg mailing list. The codecs being discussed are the same ones we’ll be including in Firefox 3.5 and are also the same codecs that Mozilla, Wikipedia and others have been investing in.

    Recent developer nightlies of Google Chrome support these codecs and a future version of Opera will also support them. Theora and Vorbis also work in Safari if you install the Xiph Qt component. We’re quickly reaching the point where all modern browsers support these open codecs with full support for the video tag.

    You’ll note that Greg’s post doesn’t have the tone of a marketing document – it’s not meant to. Nor is this a comparison against HD-sized, high-bitrate video. Instead it’s an attempt to give an honest comparison of how the open codecs fare against commonly-used formats and sizes used on the world’s largest video site. I think you’ll agree with Greg’s conclusions at the bottom of the document, especially with audio where Vorbis really shines.

    Greg’s post follows.

    Purpose

    On Jun 13th 2009 Chris DiBona of Google made a remarkable claim on the WhatWG mailing list:

    “If [youtube] were to switch to theora and maintain even a semblance of the current youtube quality it would take up most available bandwidth across the Internet.”

    Unfortunately, open video formats have been subjected to FUD so frequently that people are willing to believe bold claims like these without demanding substantiation.

    In this comparison I will demonstrate that this claim was unfair and unreasonable. Using a simple test case I show that Theora is competitive and even superior to some of the files that Google is distributing today on YouTube.

    Theora isn’t the most efficient video codec available right now. But it is by no means bad, and it is substantially better than many other widely used options. By conventional criteria Theora is competitive. It also has the substantial advantage of being unencumbered, reasonable in computational complexity, and entirely open source. People are often confused by the correct observation that Theora doesn’t provide the state of the art in bitrate vs quality, and take that to mean that Theora does poorly when in reality it does quite well. Also, the Theora encoder has improved a lot lately so some older problems no longer apply.

    While different files may produce different results, the allegation made on WhatWG was so expansive that I believe a simple comparison can reliably demonstrate its falsehood.

    I do not believe Chris intended to deceive anyone, only that he is a victim of the same outdated and/or simply inaccurate information that has fooled many others. Automotive enthusiasts may make a big deal about a 5 horsepower difference between two cars, but these kinds of raw performance differences are not relevant to most car buyers nor are they even the most important criteria to people who race. Likewise, videophiles nitpick the quality of compression formats and this nitpicking is important for the advancement of the art. But I believe that people are mistaking these kinds of small differences for something which is relevant to their own codec selection.

    Results

    A 499kbit/sec H.264+AAC output and a 327kbit/sec H.263(Sorensen Spark)+MP3 output were available via the download service. The YouTube-encoded files are available on the YouTube site. Because the files on YouTube may change and the web player does not disclose the underlying bitrate, I have made the two encoded files available.

    ~499kbit/sec comparison

    YouTube

    Download (H.264+AAC; 17MiB)

    Ogg/Theora+Vorbis

    Download / Watch (Ogg/Theora+Vorbis; 17MiB)

    ~327kbit/sec comparison

    YouTube

    Download (H.263+MP3; 12MiB)

    Ogg/Theora+Vorbis

    Download / Watch (Ogg/Theora+Vorbis; 12MiB)

    A slightly lower bitrate was used for the Theora+Vorbis test cases to avoid any question of quality improvement resulting from larger outputs.
    For a fair comparison you must compare the audio as well. Even without audio differences, still image comparisons are a poor proxy for video quality.
    I provided this random frame still image comparison only because I expect that people will not bother watching the examples without evidence that the results are interesting.

    Methodology

    In order to avoid any possible bias in the selection of H.264 encoders and encoding options, and to maximize the relevance for this particular issue, I’ve used YouTube itself as the H.264 encoder. This is less than ideal because YouTube does not accept lossless input, but it does accept arbitrarily high bitrate inputs.

    I utilized the Blender Foundation’s Big Buck Bunny as my test case because of its clear licensing status, because it’s a real world test case, and because I have it available in a lossless format. I am not aware of any reason why this particular clip would favor either Theora or H.264.

    I chose to use a test case with a soundtrack because most real usage has sound. No one implements HTML5 video without audio, and no one is implementing either of Theora or Vorbis without the other. Vorbis’s state-of-the-art performance is a contributor to the overall Ogg/Theora+Vorbis solution.

    • Obtain the lossless 640×360 Big Buck Bunny source PNGs and FLACs from media.xiph.org.
    • Resample the images to 480×270 using ImageMagick’s convert utility.
    • Use gstreamer’s jpegenc, produce a quality=100 mjpeg + PCM audio stream. The result is around 1.5Gbytes with a bitrate of around 20Mbit/sec.
    • Truncate the file to fit under the YouTube 1Gbyte limit, resulting in input_mjpeg.avi (706MiB).
    • Upload the file to YouTube and wait for it to transcode.
    • Download the FLV and H.264 files produced by YouTube using one of the many web downloading services. (I used keepvid)
    • Using libtheora 1.1a2 and Vorbis aoTuv 5.7 produce a file of comparable bitrate to the youtube 499kbit/sec from the same file uploaded to YouTube (input_mjpeg.avi).
    • Resample the file uploaded to YouTube to 400×226.
    • Using libtheora 1.1a2 and Vorbis aoTuv 5.7 produce a file of comparable bitrate to the youtube 327kbit/sec from the 400×226 downsampled copy of input_mjpeg.avi.

    I later discovered that YouTube sometimes offers additional sizes. I tried the youtube-dl utility and it indicated that these other sizes were not available for my file. Otherwise I would have also included them in this comparison.

    A keyframe interval of 250 frames was used for the Theora encoding. The theora 1.1a2 encoder software used is available from theora.org. The Vorbis encoder used is available from the aoTuV website. No software modifications were performed.

    My conclusions

    It can be difficult to compare video at low bitrates, and even YouTube’s higher bitrate option is not high enough to achieve good quality. The primary challenge is that all files at these rates will have problems, so the reviewer is often forced to decide which of two entirely distinct flaws is worse. Sometimes people come to different conclusions.

    That said, I believe that the Theora+Vorbis results are substantially better than the YouTube 327kbit/sec. Several other people have expressed the same view to me, and I expect you’ll also reach the same conclusion. This is unsurprising since we’ve been telling people that Theora is better than H.263, especially at lower bitrates, for some time now and YouTube only uses a subset of H.263.

    The low bitrate case is also helped by Vorbis’ considerable superiority over MP3. For example, the crickets at the beginning are inaudible in the low rate YouTube clip but sound fine in the Ogg/Theora+Vorbis version.

    In the case of the 499kbit/sec H.264 I believe that under careful comparison many people would prefer the H.264 video. However, the difference is not especially great. I expect that most casual users would be unlikely to express a preference or complain about quality if one was substituted for another and I’ve had several people perform a casual comparison of the files and express indifference. Since Theora+Vorbis is providing such comparable results, I think I can confidently state that reports of the internet’s impending demise are greatly exaggerated.

    Of course, YouTube may be using an inferior processing chain, or encoding options which trade off quality for some other desirable characteristic (like better seeking granularity, encoding speed, or a specific rate control pattern). But even if they are, we can conclude that adopting an an open unencumbered format in addition to or instead of their current offerings would not cause problems on the basis of quality or bitrate.

    But please— see and hear for yourself.

  7. Shiva – More than a RESTful API to your music collection

    Music for me is not only part of my daily life, it is an essential part. It helps me concentrate, improves my mood, distracts me and/or helps me relax. This is true for most (if not all) people.The lack of music or the wrong selection of tunes can have the complete opposite effect, it has a strong influence on how we feel. It also plays a key role in shaping our identity. Music, like most (if not all) types of culture, is not an accessory, is not something we can choose to ignore, it is a need that we have as human beings.

    The Internet has become the most efficient medium ever in culture distribution. Today it’s easier than ever to have access to a huge diversity of culture, from any place in the world. At the same time you can reach the whole world with your music, instantly, with just signing up at one of the many websites you can find for music distribution. Just as “travel broadens the mind”, music sharing enriches culture, and thanks to the Internet, culture is nowadays more alive than ever.

    Not too long ago record labels were the judges of what was good music (by their standards) and what was not. They controlled the only global-scale distribution channel, so to make use of it you would need to come to an agreement with them, which usually meant giving up most of the rights over your cultural pieces. Creating and maintaining such a channel was neither easy nor cheap, there was a need for the service they provided, and even though their goal was not to distribute culture but to be profitable (as every company) both parties, industry and society, benefited from this.

    Times have changed and this model is obsolete now; the king is dead, so there are companies fighting to occupy this vacancy. What also changed was the business model. Now it is not just the music – it is also about restricting access to it and collecting (and selling) private information about the listeners. In other words, DRM and privacy. Here is where Shiva comes into play.

    What is Shiva?

    Shiva is, technically speaking, a RESTful API to your music collection. It indexes your music and exposes an API with the metadata of your files so you can then perform queries on it and organize it as you wish.

    On a higher level, however, Shiva aims to be a free (as in freedom and beer) alternative to popular music services. It was born with the goal of giving back the control over their music and privacy to the users, protecting them from the industry’s obsession with control.

    It’s not intended to compete directly with online music services, but to be an alternative that you can install and modify to your needs. You will own the music in your server. Nobody but you (or whoever you give permission) will be able to delete files or modify the files’ metadata to correct it when it’s wrong. And of course, it will all be available to any device with Internet connection.

    You will also have a clean, RESTful API to your music without restrictions. You can grant access to your friends and let them use the service or, if they have their own Shiva instances, let both servers talk to each other and share the music transparently.

    To sum up, Shiva is a distributed social network for sharing music.

    Your own music server

    Shiva-Server is the component that indexes your music and exposes a RESTful API. These are the available resources:

    • /artists
      • /artists/shows
    • /albums
    • /tracks
      • /tracks/lyrics

    It’s built in python, using SQLAlchemy as ORM and Flask for HTTP communication.

    Indexing your music

    The installation process is quite simple. There’s a very complete guide in the README file, but I’ll summarize it here:

    • Get the source
    • Install dependencies from the requirements.pip file
    • Copy /shiva/config/local.py.example to /shiva/config/local.py
    • Edit it and configure the directories to scan
    • Create the database (sqlite by default)
    • Run the indexer
    • Run the development server

    For details on any of the steps, check the documentation.

    Once the music has been indexed, all the metadata is stored in the database and queried from it. Files are only accessed by the file server for streaming. Lyrics are scraped the first time they are requested and then cached. Given the changing nature of the shows resource, this is the only one that is not cached; instead is queried every time. At the moment of this writing is using only one source, the BandsInTown API.

    Once the server is running you have all you need to start playing with Shiva. Point to a resource, like /artists, to see it in action.

    Scraping lyrics

    As mentioned, lyrics are scraped, and you can create your own scrapers for specific websites that have the lyrics you want. All you need is to create a python file with a class inheriting from LyricScraper in the /shiva/lyrics directory. The following template makes clear how easy it is. Let’s say we have a file /shiva/lyrics/mylyrics.py:

    From shiva.lyrics import LyricScraper:

    class MyLyricsScraper(LyricScraper):
        “““ Fetches lyrics from mylyrics.com ”””
     
        def fetch(self, artist, title):
            # Magic happens here
     
            if not lyrics:
                return False
     
            self.lyrics = lyrics
            self.source = lyrics_url
     
            return True

    After this you need add your brand new scraper to the scrapers list, in your local.py config file:

    SCRAPERS = {
        ‘lyrics’: (
            ‘mylyrics.MyLyricScraper,
        )
    }

    Shiva will instantiate your scraper and call the fetch() method. If it returns True, it will then proceed to look for the lyrics in the lyrics attribute, and the URL from which they were scraped in the source attribute:

    if scraper.fetch():
        lyrics = Lyrics(text=scraper.lyrics, source=scraper.source,
                        track=track)
        g.db.session.add(lyrics)
        g.db.session.commit()
     
        return lyrics

    Check the existing scrapers for real world examples.

    Lyrics will only be fetched when you request one specific tracks, not when retrieving more than one. The reason behind this is that each track’s lyrics may require two or more requests, and we don’t want to DoS the website when retrieving an artist’s discography. That would not be nice.

    Setting up a file server

    The development server, as its name clearly states, should not be used for production. In fact it is almost impossible because can only serve one request at a time, and the audio element will keep the connection open as long as the file is playing, ergo, blocking completely the API.

    Shiva provides a way to delegate the file serving to a dedicated server. For this you have to edit your /shiva/config/local.py file, and edit the MEDIA_DIRS setting. This option expects a tuple of MediaDir objects, which provide the mechanism to define directories to scan and a socket to serve the files through:

    MediaDir(‘/srv/music’, url=’http://localhost:8080)

    This way doesn’t matter in which socket your application runs, files in the /src/music directory will be served through the URL defined in the url attribute. This object also allows to define subdirectories to be scanned. For example:

    MediaDir(‘/srv/music’, dirs=(‘/pop’, ‘/rock’), url=’http://localhost:8080)

    In this case only the directories /srv/music/pop and /srv/music/rock will be scanned. You can define as many MediaDir objects as you need. Suppose you have the file /srv/music/rock/nofx-dinosaurs_will_die.mp3, once this is in place the track’s download_uri attribute will be:

    {
        "slug": "dinosaurs-will-die",
        "title": "Dinosaurs Will Die",
        "uri": "/track/510",
        "id": 510,
        "stream_uri": "http://localhost:8080/nofx-dinosaurs_will_die.mp3"
    }

    Your own music player

    Once you have your music scanned and the API running, you need a client that consumes those services and plays your music, like Shiva-Client. Built as a single page application with AngularJS and HTML5 technologies, like Audio and Drag and Drop, this client will allow you to browse through your catalog, add files to a playlist and play them.

    Due to the Same-Origin-Policy you will need a server that acts as proxy between the web app and the API. For this you will find a server.py file in the repo that will do this for you. The only dependency for this file is Flask, but I assume you have that installed already. Now just execute it:

    python server.py

    This will run the server on http://localhost:9001/

    Access that URI and check the server output, you will see not only the media needed by the app (like images and javascript files) but also a /api/artists call. That’s the proxy. Any call to /api/ will be redirected by the server to http://localhost:9002/

    If you open a console, like firebug, you will see a Shiva object in the global namespace. Inside of it you will find 2 main attributes, Player and Playlist. Those objects encapsulate all the logic for queueing and playing music. The Player only holds the current track, acts as a wrapper around HTML5’s Audio element. What may not seem natural at first is that normally you won’t interact with the Player, but with the Playlist, which acts as a façade because it knows all the tracks and instructs the Player which track to load and play next.

    The source for those objects is in the js/controllers.js file. There you will also find the AngularJS controllers, which perform the actual calls to the API. It consists of just 2 calls, one to get the list of artists and another one to get the discography for an artist. Check the code, is quite simple.

    So once tracks are added to the playlist, you can do things like play it:

    Shiva.Playlist.play()

    Stop it:

    Shiva.Playlist.stop()

    Or skip the track:

    Shiva.Playlist.next()

    Some performance optimizations were made in order to lower the processing as much as possible. For example, you will see a progress bar when playing music, that will only be updated when it has to be shown. The events will be removed when not needed to avoid any unnecessary DOM manipulation of non-visible elements:

    Shiva.Player.audio.removeEventListener('timeupdate',    Shiva.Player.audio.timeUpdateHandler, false);

    Present and future

    At the time of this writing, Shiva is in a usable state and provides the core functionality but is still young and lacks some important features. That’s also why this article doesn’t dig too much into the code, because it will rapidly change. To know the current status please check the documentation.

    If you want to contribute, there are many ways you can help the project. First of all, fork both the server and the client, play with them and send your improvements back to upstream.

    Request features. If you think “seems nice, but I wouldn’t use it” write down why and send those thoughts and ideally some ideas on how to tackle them. There is a long list of lacking features; your help is vital in order to prioritize them.

    But most importantly; build your own client. You know what you like about your favourite music player and you know what sucks. Fork and modify the existing client or create your own from scratch, bring new ideas to the old world of music players. Give new and creative uses to the API.

    There’s a lot of work to be done, in many different fronts. Some of the plans for the future are:

    • Shiva-jslib: An easy to use javascript library that encapsulates the API calls, so you can focus only on building the GUI and forget about the protocol.
    • Shiva2Shiva communication: Let two (or more) Shiva instances talk to each other to allow for transparent sharing of music between servers.
    • Shiva-FXOS: A Shiva client for Firefox OS.

    And anything else you can think of. Code, send ideas, code your clients.

    Happy hacking!

  8. JSMad – a JavaScript MP3 decoder

    It always amazes me just how fast modern browsers and their JavaScript engines are. And how creative people get when trying to make things work inside a browser instead of relying on a plugin that our end users would have to install (and more importantly constantly keep up to date).

    JS MAd

    The latest thing that make me go “wow” is jsmad (source on GitHub) by Amos Wenger, Jens Nockert and Matthias Georgi. JSMad is an MP3 decoder in JavaScript!

    “So what”, you say? Well, having JSMad means that now Firefox can play MP3 files without any Flash. It also means that you can listen to MP3 in the browser without the 64bit issues on Linux. With JSMad we can dive deep into the MP3 format and not only play the song but also get information about it. It allows us to build a lot of native dj-mixers, samplers and sequencers in the nearer future.

    Right now JSMad works in Firefox 4+ and on Chrome 13.0+, if you enable the Web Audio API in ‘about:flags’.

    I remember when MP3 came out and my computer back then was too slow to encode it without locking up in WinAmp. Back then a scene player also helped me out. Now we do the same inside a browser rather than desktop applications.

  9. HTML5 video ‘buffered’ property available in Firefox 4

    This is a repost from Chris Pearce’s blog.

    Recently I landed support for the HTML5 video ‘buffered’ property in Firefox. This is cool because we can now accurately determine which time-segments of a video we can play and seek into without needing to pause playback to download more data. Previously you could only get the byte position the download had reached, which often doesn’t map to the time ranges which are playable very well, especially in a variable bit rate video. This also can’t tell you if there are chunks which we skipped downloading before the downloaded byte position. Once the video controls UI is updated, users will be able to know exactly which segments of their video are downloaded and playable and can be seeked into without pausing playback to download more data.

    To see this in action, download a current Firefox nightly build , and point your browser at my video ‘buffered’ property demo. You’ll see something like the screenshot below, including an extra progress bar (implemented using canvas) showing the time ranges which are buffered.

    I’ve implemented the ‘buffered’ property for the Ogg and WAV backends. Support for the ‘buffered’ property for WebM is being worked on by Matthew Gregan, and is well underway. At the moment we return empty ranges for the ‘buffered’ property on video elements playing WebM and raw video.

    My checkin just missed the cutoff for Firefox 4 Beta 3, so the first beta release that the video ‘buffered’ property will appear in is Firefox 4 Beta 4.