kaioa.com

Something is missing

Now and Then

ActionScript was introduced with the release of Flash 4 back in 1999. In my never-ending naïveté I assumed it should be pretty mature by now. Almost a decade should be enough for that, shouldn't it? Obviously that isn't the case - otherwise there wouldn't anything to rant about. ;)

I started with ActionScript3 (Flash9+) a few weeks ago. It's supposed to be so much better than ActionScript2. There wasn't much of a reason to not use the latest version; given that about 90% of the users use the latest version of Flash. The remaining 10% quickly fade away if you consider things like the lifespan of small web games (a couple of years easily) and the time it takes to get things rolling for real. A quick look at AS2 was actually enough to discard it completely; it's ugly.

In the past I always wondered how they managed to get the Flash VM this small. Well, now I know. They only cover absolutely basic functionality, which wasn't much of a surprise to be honest. Given the limited scope of Flash applets this isn't really a problem. However, if you take a close look at those things they did implement, you'll be able to see what makes it this lightweight: half the functionality is missing.

An Example - (HE-)AAC is superior to MP3

Adobe recently added support for H.264 (a video format), (HE-)AAC (an audio format), and MP4/3GP/MOV/etc. (container formats) to their Flash runtime. (HE)-AAC is somewhat similar to Ogg Vorbis; at low bitrates the sound gets "washed out" without introducing those painful artifacts everyone knows from low-bitrate MP3s. Hi-hats in particular are as pleasant as a million rusty nails scratching across a blackboard. With better lossy psychoacoustic audio formats the sound only gets "thinner". For a more visual example compare JPG with wavelet-based formats such as JP2 (JPEG2000). At high compression rates JPG produces heavy block artifacts whereas JP2 only looses details, which is a lot more pleasant to the eye since no noise is added.

Since you don't get those artifacts with (HE-)AAC you can go really low with the bitrate. In my tests I could get as low as ~25kbits with HE-AAC (AAC SBR+PS). To put things into perspective: that's 73kb for a 22s background music loop. An MP3 loop with similar quality requires 64kbits if you convert it to mono or 96kbits if you use joint stereo (for "real" stereo i.e. 2 channel audio you should at least use 256kbits). That's 177kb (+142%) respective 265kb (+263%). While the MP3 sounds a bit "fuller", the AAC track is more pleasant to listen to.

This difference is huge - especially if you consider that you need a million downloads per month for a mere $1000 (the eCPM is typically at about 1$ with those popular ad-wrapper networks). Let's say you got three 22s tracks (or 1 minute and 6 seconds of audio) and you actually manage to get a million downloads. Then you'll have an overhead of 312kb per download (with 64kbits that is), which means you'll waste 312gb of traffic. Of course you'll also make your visitors wait longer than necessary.

Futile workarounds

As if all of that wouldn't be enough to despise MP3, there is yet another issue: MP3 isn't suitable for looping. By design the format adds leading and trailing silence. There are tools which are supposed to work around this, but they don't work very well.

One might be inclined to try to fix it with some code like this one (don't bother trying it - it doesn't work!):

[Embed(source = 'loop.mp3')]
private var musicClass:Class;
private var music:Sound = new musicClass();
private var soundChannel:SoundChannel=null;
public function Main():void {
    [...]
    var musicTimer:Timer = new Timer(1);
    musicTimer.addEventListener(TimerEvent.TIMER, reloopMusic);
    musicTimer.start();
}
private function reloopMusic(evt:TimerEvent):void {
    if (soundChannel == null){
        soundChannel = music.play(42);
        return;
    }
    if (soundChannel.position >= music.length - 18){
        soundChannel.stop();
        soundChannel = music.play(42);
    }
}

42 is the length of the leading silence and 18 is the length of the trailing silence (both in msecs). Looks like it could work, right? Well, it doesn't work. There are two reasons why it doesn't work as expected:

1. The minimum interval of Timer depends on the SWF file's framerate and the used browser (IE 16msec, Opera 8msec, Firefox 1msec - may vary with versions and operating systems) and
2. SoundChannel.position is broken. The reported position isn't even remotely accurate.

If the Flash IDE is used you can add WAV files, enable MP3 compression, and then some magical voodoo fixes the loop problem for you. But that doesn't help much if you're using the SDK directly, does it?

(HE-)AAC to the rescue! (Or not)

Subjectively improved quality, smaller download sizes, and no looping issues. What could go possibly wrong with so much awesomeness at one spot? Well, the incomplete Flash API does.

flash.media.Sound is only capable of playing MP3 audio files

Being able to use Sound would be really nice. Like Bitmap takes JPG, GIF, and PNG; Sound should take MP3, AAC, or whatever else they decide to add in the future. Blatantly obvious, isn't it?

flash.media.Video requires a flash.net.NetStream

This means you cannot load embedded videos - or what I'm trying in this case - embedded MP4s which only contain an HE-AAC audio stream.

mxmlc isn't aware of the "video/mp4" mime type

[Embed(source = 'music/mp1.mp4', mimeType = "video/mp4")] //correct mime - doesn't work
[Embed(source = 'music/mp1.mp4', mimeType = "application/octet-stream")] //gotta use this

In a sane world...

Sound should of course be able to load audio streams from MP4 containers, Video would take an InputStream, there would be also utility streams like ByteArrayInputStream, and Embed would know about those containers and their mime types.

The functionality is there. Of course I want to use it. It's truly mind-boggling that Adobe failed to predict that. There are a lot of hobbyist APIs which are more feature complete than that. It's the sparkling new killer feature; expose it!

They also should have done something about that looping issue. E.g. optional auto trimming which defaults to on. Or even better yet a different audio format. If you look a bit around at various Flash forums you can find thousands of threads about this very topic. This single issue must have wasted massive amounts of time, energy, and money. I don't know how something like that could happen with a supposedly media centric solution like Flash. It can't get any more basic than audio loops, can it?

Needless to say that I'm very disappointed with Flash9. It should get at least the overly obvious things right.