Working with MIDI files and Flash (actionscript)

date: 30 Nov, 2010, actionscript: 3 FP10, complexity: complexity level 4.2 of 5complexity level 4.2 of 5complexity level 4.2 of 5complexity level 4.2 of 5complexity level 4.2 of 5, feedback (votes): users score 4.99 of 5 (125 votes)users score 4.99 of 5 (125 votes)users score 4.99 of 5 (125 votes)users score 4.99 of 5 (125 votes)users score 4.99 of 5 (125 votes)

About 3 years ago, when actionscript 3 was born, Andre Michelle came with the idea of injecting audio data into a ByteArray (binary SWF structure) for generating dynamic audio, I did have a go with this approach for creating my own music dynamically (on the fly) based on pulses previously stored in a MIDI file, which it worked fine but it wasn't too good in terms of performance and synchronization, especially creating long sequences by linking audio files where easily all of them get out of synch in slow machines. So I forgot about all this till Adobe has introduced into the flash player platform (FP10) the very famous SampleDataEvent.SAMPLE_DATA, giving us lots of ways for generating dynamic audio.

Dynamic audio in Flash

In my previous article speaking about "interpolation" I have made a Piano Keyboard of 6 octaves as proof of concept of reusing one single piano key by changing its pitch for generating the rest of the keys. Well, that was a good practice but it is a bit useless if we cannot have any control on it apart of clicking the keys with the mouse. A friend of mine said to me, you should enable the keyboard and map the keys, say A,S,D,F,G,H,J and use the W,E,T,Y… for semitones but that was still useless for me, I mean, the approach of it was just explaining how the hermite interpolation works and not making a synthesizer that everybody can play and feel great with.

Actually there are some online audio applications made in flash like the hobnox (audiotool) which contains many crazy synthesizers modules made natively in actionscript plus a very complex sequencer and a huge set of different effects. The first time I saw it, I said Wooooooooooowwwww but I have to confess that once I've started using it, I didn't stick to it for long and I jumped back to Cubase, Logic, Reason, FruityLoops. Forgive me if I'm making a very wrong comparison; I am not saying that it is a bad tool, No! It is perhaps the most complicated online flash software I have ever seen in my entire life which is great for the portfolio of the people who have been involved in that project (especial thanks to Andre Michelle, Joa Ebert and the people who made it, it is really brilliant, a great idea… I particularly think that the SampleDataEvent.SAMPLE_DATA exists thanks to you guys). The reason why I said that is because of I am so used to desktop audio applications where we have audio hardware acceleration ASIO, DSP, midi interfaces, very fast UI response, plus lots, lots of resources, thousands of people making plugins, effects, synths, etc. That's why I found difficult to feel familiarized with the audiotool, I think it is still in progress and perhaps in the future it will become an online audio tool as good as any other desktop audio software, the last time I saw it was promising a lot, I can imagine how complicated this flash application is.

From Fruity Loops to ActionScript

Well, unfortunately there is not an easy way of doing this transparently, import an audio project into actionscript from an audio software is something very complicated, apart of generating lots of WAVs of course. Especially all the audio that is being processed dynamically by effects, synthesizers modules, etc. Most of these audio programs used hardware acceleration and against that we can't do much. I have tried many different ways to import my audio projects into Flash but I always end up with exporting lots of WAVs which is not good in terms of swf-file-size, and unfortunately It seems to me that this step is unavoidable unless you want to make your own synthesizers in actionscript natively, similar they did it in the audiotool (crazy).

The middle term

Well the middle term is reusing as much as we can waveforms (.wav) and apply some easy (non expensive in terms of CPU) effects on top of them, so we can have at the end a good sequence filled of good samples, change the pitch of them on the fly, with some filters, etc… it is not too bad! And it is still CD quality and it sounds good! And the best part of all this is that it will sound the same in all computers, in all platforms (the flash player platform magic).

Playing Midi Files

Well , some people don't know exactly what midi is, and some of them are very confused about General Midi and MIDI itself. Midi means pulses (messages), it is a communication protocol used in many different areas (not only in music), did you know that even some medical equipments at hospitals use midi? MIDI is a very light, old and solid protocol for communicating computers, mixers, consoles, musical instruments, etc. find out more about it at www.midi.org

Now, General MIDI is something 100% related to music, it consists in a set of musical instruments (128 in total) which includes some traditional instruments such as piano, guitar, flute, trombone, trumpet, etc… and also it includes a Helicopter sound, a gunshot, stadium crowd, etc… don't ask me why. First time I saw General Midi was when I bought a workstation keyboard earlier on 90's (KORG X3). I used General Midi by that time and then I was a bit disappointed when I tried to play my songs later on in another keyboard (Roland XP50). It sounded completely different, a different thing.

General midi is a good approach if what you want to do is show an idea about how a song sounds like, it uses the MIDI messaging protocol for playing each of the set of 128 instruments, MIDI lack of sounds (it is only messages), all these instruments are implemented internally by the device (machine) and played when processing MIDI messages. It has a big disadvantage and it is that it doesn't guarantee that your music will play and sound the same in all devices (platforms), so there is not fidelity at all by using General Midi format. It is like HTML earlier on 90's, it looks different between browsers, General MIDI sounds different between devices and all this is sometimes annoying.

So, let's forget a bit General MIDI and focus in the important part, the MIDI communication protocol. Please take a look on the following video for understanding a bit more how all this works in Flash

and then, let's jump to the examples at the second page

page #1 | page #2 | page #3

Resources

A very interesting research about recording acoustics http://www.yoambulante.com/en/labs/pagodinho.php

Finally I got the chance for writing some documentation about all this MIDI files research in flash.

check the online detailed documentation Midi Files Reader
some demos midifiles examples, feel free to ask if you have any doubt.

date: 03 May 2012 - 20:46:00, published by javier
Great Work!
date: 13 Jul 2011 - 11:29:09
lol!! awesome, the MarioFortressSNES looks amazing.
date: 18 Apr 2011 - 13:18:30, published by Marc
Excellent! Thanks for sharing. . . Since we lost the Sequence Xtra for Director I've been looking for a Flash-based solution for playing and controlling MIDI. Currently using a Flash object in MAX/MSP but it's kludgy and a pain to bundle for distribution.
date: 06 Dec 2010 - 09:51:49, published by Leandro Zanol
Great article! Nice job! I'm working on some voice recognition (still freq only) games for music teaching and I'm on my college last year (2011) and doing my final work (don't know its name! :P) on the FFT and HMM algorithms... as a proof of concept too, I'll try building a voice command navigable website/app, let's see... Greetings!
date: 30 Nov 2010 - 22:56:33, published by yohami
unbelievable, x10