Search This Blog

Showing posts with label MIDI. Show all posts
Showing posts with label MIDI. Show all posts

Tuesday, July 26, 2011

Synthodeon, MIDI and the Future... (Part 2)

So far I have not said anything remotely interesting, at least technically, but hopefully that will change now... 

There are in fact products just like I described in my last post: ipMIDI, a C# MIDI code set, a Cycling 74 patch, QmidiNet, and many others.  All of these products, though, treat MIDI as a simple end-point-to-end-point topology that does nothing beyond the basic MIDI capabilities.

But my interest here goes beyond this basic ability.

First off, when you are working with software samplers, e.g., Kontakt, you have the ability to organize multiple samplers on the same MIDI channel or duplicate samplers on a set of MIDI channels.  This ability allows you to have much finer control over what is played for a given set of MIDI values as well as allows you to do things that you otherwise could not.

In terms of something like, say, Guitar samples, its often the case that there are a lot of articulations available for a given sound, e.g., strumming up versus strumming down, starting on one string, ending on another, mutes, harmonics, and so on.  Many software samplers today cram these sounds into chord sequences, e.g., I hold down "C" in on section of keys and I use "A#" in another to strum a C chord.  Some samplers allow you to control what chords are being sounded (on the fly or with setups) and others do not.

What happens is that the complexity of what is being (or needs to be played) quickly and exponentially gets out of hand - there are too many ways (inversions, where on the "guitar neck" your hand is, and so on) to play a "C" chord, which form of "C" is appropriate for the piece I am playing, and so on.

At a low level MIDI is getting the "note up & down" events to the sampler but the "sound" the musician is asking for is somewhat beyond this basic ability.

While things like MIDI CC values could be used to address this issue they are not generally because picking you own choices for specific CC values may be "non-standard" and would render any software or hardware that did so incompatible with a lot of existing MIDI equipment.

Even something somewhat standard, like breath control (velocity) MIDI values, are not always handled correctly.  For example, I've played an EWI 4000s live for several years using either Mr. Sax (from samplemodelling.com) or Roland XV-5050s with the Patchman Sound banks (BTW, I cannot say enough good things about these sound banks).  Some out-of-the-box 5050 patches works with a breath controller, some do not (patchman ones always do, even on the XV-5050, as they were designed for this purpose).

Mr. Sax uses a variety of CC controls for special things while playing - which is inconvenient if you do not have a way to generate them.  This is particularly true live - I have enough to haul around as it is without yet more knobs and pedals to munge CC values.

For the more sophisticated guitar functionality you would like a controller to be able to do "more" for the player in terms of triggering samples in response to the player's hand movements.  There are many cases where one MIDI channel is simply not enough to do this.

Take, for example, note bends.  On a synth keyboard you typically bend the whole keyboard with the pitch wheel, i.e., all the notes change in response to the wheel.  On a faux sax or clarinet with the EWI you bend a single note (by biting the mouth piece).  On a guitar you can bend multiple notes differently and simultaneously.  MIDI does not help you in this regard.  (Not to beat up MIDI - it was developed before the things I am talking about were even possible.)

Now throw into the mix your typical keyboard layering and/or splits.  Now I really need multiple MIDI channels per split to do anything beyond basic notes.

Then there are devices like the iOS ones from Apple (and well, possibly, as Android).  Playing them wireless in the context of a guitar or keyboard where exact millisecond  timing is required is a show stopper - its simply not reliable enough.

(Yes you can create a private network on your Mac, etc. but I've not had satisfactory results much less taken it into a paid live gig alone or with other musicians.)

You really need a wire to get this done.

(At least Macs work live - I've played a variety of samplers and Logic live without problems.  I guess the same is true for PCs but I personally would not rely on one...  BTW, I am no PC bigot and what I will be posting going forward will certainly apply to PCs as well.)

So the bottom line here is that we need more than just a single MIDI channel between a controller for modern samplers if we want to get anything done that gives us much closer control that we get for a single keyboard, etc.

My solution to this is very, very simple: combine collections of MIDI channels into (sub-)groups, or what I will refer to as simply groups and sets of MIDI virtual ports into what I will call Multis.

Now MIDI makes this easy on a computer because ports have names, e.g., Kontakt Virtual Output.  These names are derived from the software or, in the case of hardware, from the device itself as part of the USB standard for modern MIDI devices.

Devices are often, even in the wild, named sequentially with simple numbering, e.g., E-MU Xmidi 2x2 Midi Out 1, E-MU Xmidi 2x2 Midi Out 2, ..., so it makes sense to think of sequentially numbered devices as a Multi.

But fortunately we don't have to worry about actual devices for this abstract model.  If we have software that simply creates groups of ports through the local OS we can create and name them any way we like, e.g., Port 1, Port 2, Port 3.

Secondly, because in the world I describe we can support any number of MIDI port plus have a fast, reliable means to talk between the we can have a relatively large number of MIDI channels in a Multi, say four, and still achieve very high performance.

So, for example, in the diagram below we have "A MIDI 1" a (Single) logical OS-based MIDI port we've created on Node 1 that's linked to Node 2 (or to other random Nodes).  We also have a set of MIDI ports "B MIDI 1", "B MIDI 2", and "B MIDI 3" that we want to think about as a Multi.  We name this Multi by dropping off the space and number to get "B MIDI".  Any port the follows the template of "B MIDI x" where x is a sequential number is included.



We divide "A MIDI 1" up into groups of, for example, four MIDI channels - groups being named 1, 2, 3, and 4.  We also divide the Multi "B MIDI" into twelve groups of four MIDI channels.  Groups are numbered from one on the "first" MIDI port and consume units of n channels per group until you run out of channels (n is always less than 16 - at least for now - though I suppose there is no reason n couldn't be 32 or 18 or something else like that).

So what we have done is create logical groups of MIDI channels into a simple grouping and naming system.  There's no requirement as to how many or few channels fit into a group, or how many ports into a Multi - those are decisions based on the application.

So what?

You might love MIDI as it is and hate me for what I am saying - so be it.

But face it, as I said in the last post, MIDI is like a Platypus - its here, it works, we love it but its not the best we can do and its certainly not the latest design...

For one thing when we talk to a sampler it makes more sense to talk about a sound using a group rather than a channel.  (Certainly one sound can be one group which is one channel - which is what MIDI is today.)  But if I want to create a more complex controller that works like some of the guitar-based sampler keyboard inputs I can (no pun intended) re-group into a set of, say four or eight MIDI channels, and use the extra bandwidth to do interesting things in the controller to diddle the sampler in interesting ways.

For another we can send metadata along a given Multi between apps that speak the same MIDI protocol at the Multi level.  Because I have good hard bandwidth between, say, Node 1 and Node 2, I can encode things into MIDI and use it as a transport layer - either in a group or as a Multi.  (There are other, more serious reasons to want to do this which will be revealed as the '3i' gets closer to debut.)  This gives me a reason not to invent yet another MIDI over UDP/TCP protocol - though I may release tools that make of this simple to do.

After all this is basically kind of like what's going on with MIDI-controlled DAW control surfaces so why not buy into the same concept for actual MIDI controllers?

In the reality of the 3i what this all really means is that the 3i thinks about the world as groups of MIDI channels - not just one.  Which makes effects beyond what one channel can do possible if not down right easy.   The 3i sort of kind of only cares about output groups and its set involves knowledge of the target sampler to some degree in order to take advantage of what the sampler can really do.

Synthodeon, MIDI and the Future... (Part 1)

As a developer of software for the last 35 odd years and a gigging musician for the last 10 or so years I have come to know MIDI quite well.

Recently Synthodeon has been working on something called the '3i'.  This is, among other things, a MIDI controller that will allow new types of playing that currently impossible due to the fundamental limitations of MIDI.  While I cannot yet talk about exactly what the 3i is and does I can say a few things about what it needs to do MIDI-wise.

First of all MIDI has been tied to the notion of a MIDI cable and to some extent a "keyboard" - after all this is what it was originally designed for.  There are sixteen channels on the cable and each channel supports things like pitch wheels, notes and all the standard MIDI stuff.   Most of today's keyboard controllers support other things like CC messages for knobs and sliders.

But on the whole the "concept" is still very simple: Channels are tied to sound controllers (maybe with splits using extra MIDI channels to support two different sounds or to overlay sounds).  Keys pressed operate within a given channel using Note On/Off messages, and so on.

So what don't I like about MIDI?

First off I don't like the basic model because its so very limited - particularly when you look at what can be done with things like Kontakt or Logic or any other MIDI-able DAW-type software.  These apps have the ability to tie different samplers or functions to specific MIDI channels, i.e., I can have the same sampler on three consecutive MIDI channels.  But, for example, most keyboards can only access a single sampler.

Secondly certain types of MIDI software in Kontakt, e.g., guitars in Orange Tree or Vir2, have funky multi-key controller options for their samples, e.g., you have keys to select a chord or note and separate "strum" or other note-sounding keys, reminiscent an AutoHarp.  This is somewhat awkward and pushes the notion of MIDI away from keyboards - but there is no alternative place to go currently.

Third the MIDI bend (and in general the CC) model is very limited - I can only perform functions at the channel level.  What happens if I want to bend a single note?  Controllers I am aware of don't handle this at all.

Fourth, with the advent of network software (wireless, wired, USB, etc.) MIDI can run over a virtual channel from point A to point B.  However there is no standard model for this.  There is stuff like OSC and others but they are not standardized.

Fifth, things like iOS devices have limited asymmetric MIDI functionality.  What I mean by this is that on an iPad, for example, you can plug in a MIDI device but only through a "Camera Connector" - and then your device must meet a variety of vague requirements (USB and power) in order to work.

Sixth MIDI is just not reliable enough for serious live music over WiFi regardless of what you do - even on local, private WiFi's with Mac's.

Seventh, I have a lot of devices - computers, iOS devices, keyboards, wind instruments.  I want a MIDI model that represents each of these "places" on some sort of virtual "MIDI backbone" so that I can plug something into one device on the backbone and have that connection available at other points on the backbone.

Now I am sure there are a lot of arguments about why I am wrong, or why such-and-such a specific tool can overcome one of the limitations I listed above, but the bottom line is MIDI is twenty-some years old and really hasn't kept up with modern times.  I want something that works in a modern way.

(Can you think of anything else you use that old on a daily computer or music basis save something like a physical guitar?)

I can't.

The oldest stuff I have that I use all the time are things like studio monitors and vintage synths - and most of that is not twenty years old.

So I've been looking around and found this site (OpenMuse) which talks somewhat about this in general.  There are links to some standards efforts which mostly appear to no longer exist.  There are also some patent links to US Patents claiming some form of jurisdiction over MIDI over networks.

Irrespective of any "standards" efforts there are a couple of patents to note.  First there is U.S. Patent 5,983,280 "System Using Standard Ethernet Frame Format for Communicating MIDI Information Over an Ethernet Network" and U.S. Patent 6,353,169 "Universal Audio Communications and Control System and Method."

The first patent ('280) has a UK assignee.  The patent discusses MIDI using Ethernet broadcast of various forms, multicast, and so on and it appears to be protecting some proprietary hardware and protocols.  (See this for a reasonable discussion with which I would concur.)

The second patent, owned by Gibson, uses MIDI to control audio.  Again I do not see this as relevant here.

So neither of these patents from my reading address what I will be describing here (though I am not an attorney and my opinions are worth exactly what you have paid to read article). 

Are there more patents in this area?  Its hard to say (and this is a reason the US Patent system is flawed - there is no way to discover what else is out there for a given topic).

The fact that there are numerous software apps (various OSC things, etc.) out there doing this sort of thing in one fashion or another leads me to believe that if there are they are probably not valid or everyone out there would be getting stomped on.

All that said what I want to see a MIDI software topology, at least at an abstract level, is something like this:


Here each "Node" (Node 1, Node 2, etc.) is a physical device (an iOS device, a computer, etc.).  The MIDI ports listed on each device are virtual, i.e., not physical.  If I need to link things to the outside world on a given device I can use some sort of MIDI port connector or bridge software to do that.

I want fast, clean delivery of MIDI messages - plain and simple.  No other fancy nonsense to get in the way, no wizards, no autoconnect please press okay.  (Who wants their shit to go dead during their epic solo?  Worse, who want's to thrash through menus and crap trying to fix it afterword while the audience and your band mates stare at you with hate-filled grimaces.)

Basically I want to create MIDI endpoints between devices easily, reliably, deterministically and cleanly.

I want a physical wire (green arcs) between the Nodes (USB speed or better) because I want sub-millisecond response (WiFi is okay for testing and noodling around but not for live).  To play live you need this along with the headroom it offers (for really fast pickers).

(I measured my response via MIDI on moderately fast piano piece - I can hear down to about 2.5ms - 3ms between chord strikes.  I am not a pro and I cannot play really fast but my ears are pretty good.  Sub-millisecond leaves a factor of about 2x which should be enough.)

I don't want automatic or automagic connectivity via something like Bonjour because for live gigs I don't want software "helping me" set things up.  I don't really want to (accidentally) be on the venue WiFi (where some clown will start downloading porn videos to his iPhone just as I hit the first note in my solo).

Instead I want hard coded/wired links via fixed device-based TCP/IP settings over deterministic, private connections that stay the way I set them up and come back up spot-on. (Who hasn't had the circuit breaker or generator go out in the middle of a gig at a festival?)

I don't know if a local BlueTooth network is fast enough for this - especially outside of a "computer deveice", e.g., iOS.  I have done a lot of testing and so far all I can say is "I am not sure."  I am also concerned about BlueTooth interference (audience members with lots of active BlueTooth gear on).

From a device perspective Mac OS X works reliably on stage - at least in terms of no OS, audio or midi glitches and no battery fails - networking and iOS wise this all remains to be seen.  I do not yet know about Android for this type of application though from what I have been reading its still immature in the developer area.