From alsa-devel-owner@alsa.jcu.cz  Sun May  3 23:20:24 1998
Received: from altrade.nijmegen.inter.nl.net (altrade.nijmegen.inter.nl.net [193.67.237.6])
	by marvin.jcu.cz (8.8.8/8.8.8) with ESMTP id XAA20825
	for <alsa-devel@jcu.cz>; Sun, 3 May 1998 23:20:21 +0200
Received: from obelix.fvdpol.inter.nl.net by altrade.nijmegen.inter.nl.net
	via bd99-15.Breda.NL.net [193.79.246.240] with ESMTP
	id XAA04943 (8.8.8/3.27); Sun, 3 May 1998 23:20:09 +0200 (MET DST)
Received: (from frank@localhost) by obelix.fvdpol.inter.nl.net (8.8.7/8.7.3) id AAA11745; Sun, 3 May 1998 00:22:15 +0200
From: Frank van de Pol <F.K.W.van.de.Pol@inter.nl.net>
Message-Id: <199805022222.AAA11745@obelix.fvdpol.inter.nl.net>
Subject: Re: New Sequencer core
To: P.J.Leonard@bath.ac.uk (P.J.Leonard)
Date: Sun, 3 May 1998 00:22:15 +0200 (MET DST)
Cc: alsa-devel@jcu.cz (alsa), erik@inter.nl.net (Erik van de Pol)
In-Reply-To: <35499430.167E@bath.ac.uk> from "P.J.Leonard" at May 1, 98 10:21:52 am
Content-Type: text
Reply-To: alsa-devel@alsa.jcu.cz
Sender: alsa-devel-owner@alsa.jcu.cz
Precedence: list


Hi Paul, 

here's some more thoughts for our 'melting pot'...

Frank.


P.J.Leonard wrote:
> > I'll put my (new/changed/polished) ideas next to your comments. Perhaps
> > later I'll beef up a version 0.02 of the proposal :-).
> 
>  Great it would be nice to get your doc on the web for reference ?

I'll put some stuff on my web page in about 1 or 2 weeks.

> 
> > >  I think we may need several schedulers to allow different clients to
> > > vary
> > > their tempos independently.
> > >
> > > - 2 users may want to use the system. I do not want my tempo to be
> > > changed by another.
> > 
> > Good point.
> > 
> > The architecture was designed as a 'single user' system, like is common with
> > musical workstations (and other sequencer systems). In this single user
> > environment it's an advantage to have a single 'master' time/song position.
> 
>  Yep.
>  
> > If there's a need to allow multiple independent (with regarard to timing)
> > concurrent applications; this scheme can be expanded by adding a priority
> > queue and 'master clock' (the "Timing" block from above diagram) per 'user'.
> > The addressing of message will have to be extended with an additional field
> > that determines in which priority queue (or perhaps even queues???) an event
> > has to be put.
> 
>  Yep.


I personally do not see many applications for the 'multi user' architecture,
but having a framework that allows to route events to more than one queue
(for the multi user system) gives us the possibility to distribute sequencing
over multiple boxes!!! This is definitly something that should not be
started with, but it is good to know that the architecture allows us to
build a whole sequencer cluster somewhere in the future.

So I think "YES", the system should _allow_ multiple queue's and timers
(basicly this are multiple sequencer systems), but we can start we
supporting only one queue, and add the multiple systems (and perhaps allow
distribution over multiple nodes) later.

For terminology: 

System = a sequencer Queue + Timer within the ALSA sequencer.
Node   = a computer, that is running the ALSA sequencer.

One ALSA sequencer core can have multiple 'Systems'.

----------------------

> 
> 
> > > - Another situation is the mixing of tempo and real time events. A
> > > sequencer may be
> > > playing a midi tune using the tempo but special effects applied to an
> > > individual
> > > note may require to be scheduled 80ms after the note on event.
> > >
> > 
> > I also had some thoughts in this issue. When implementing a priority queue,
> > we need to be able to compare time stamps of events. If there is a constant
> > tempo, that can easily be done. But when a tempo is changed, the number of
> > clock ticks (ppq) per second changes, and events in the queue will not have
> > correct ordering anymore. To avoid this problem, only one type of timestamp
> > is needed. We have basicly two options to choose from:
> > 
> > 1) real/clock-time, expressed in seconds (or fractions of it of course,
> >    eg. ms or us.
> > 
> > 2) songposition, expressed in clock ticks, which are related to the tempo
> >   the song is playing. for an internal resolution of eg. 1920 PPQ (parts per
> >   quarter noter), and a tempo of 135 BPM, there are 1920*135=259200 clock ticks
> >   per minute = 4320 ticks per second.
> > 
> > Because of the specific character of music (ie. it has tempo, groove etc.)
> > the second one is prefered. Conversion between these two types is trivial
> > once the currently active tempo and timestamp (in both units!) of last tempo
> > change is known.
> 
>  Yes I started off my project with the first. When I implemented real
> time changing of
> tempo a quickly changed to number 2. I was thinking of having 2 queues
> one for real time
> and one for tempo ?  The tempo queue would sit on top of the real time
> queue so you could
> ask the real time of a tempo event ?


I thought my plan over, and only supporting either of two types seems to be
always a compromise. Certain timing information like MTC and SMPTE is based
on real-time, and pumping it through an songposition based queue while the
tempo changes will result in the timing event being delivered at the wrong
time...

You just mentioned the magic word: "...having 2 queues...". This concept
will leave the decision what type to use to the user or client application. 
The master clock will have to keep the current time AND songposition; and we
can just use two priority queues in parralel. One for the 'clock time'
events, and one for the 'song position' events. Having a tempo change will
then NOT upset event ordering. If an event is enqueued by an client, we can
simply look at the field that determines the type of timestamp, and enqueue
it in the appropiate queue.

With this new feature, combined with the event routing for multiple systems,
the architectual picture looks like something like:

(Yummy, yet some more ascii art :-)


    [select on system]
 +-->-----O---->-------O------->--.........>.. 
 |        |            |  (to other systems, queue/timer combinations)
 |        V            :  (future)
 |        |
 |        O [select on timestamp type]
 |       / \
 |      /   \
 |     |     |
 |     V     V
 |   +---+ +---+
 |   | P | | P |    Priority Queue 1 = clock time
 |   | r | | r |    Priority Queue 2 = song position
 |   | i | | i |
 |   | o | | o |
 |   | Q | | Q |
 |   | 1 | | 2 |
 |   +---+ +---+
 |     |     |
 |     V     V  Dispatch event, triggered by clock    dispathers from other 
 |   +---+ +---+       +--------------+               systems on this node
 |   | D | | D |  <=== | Master clock |             | (future)
 |   +---+ +---+       +--------------+             +---+
 |     |     |                                        |    |
 |     V     V                                        V    V
 |  +---------------------------------------------------------+
 |  | Event Router                                            |
 |  +---------------------------------------------------------+
 |       |      |    |        |     |     |
 |       V      V    V        V     V     V
 |    [Client] [C]  [C]      [C]   [C]   [C]
 |       |      |    |        |     |     |
 +-<-----+---<--+-<--+----<---+--<--+--<--+





Btw: I used something similar (time stamps in two formats) in a sequencing 
program:


// class capable of storing time stamps, stores either absolute time (...s)
// or midi tick. 
class EvTime {

public:
        typedef enum {
                Tm_MIDI_Tick = 0,
                Tm_Abs_Time = 1
        } Time_t;

private:
        // should pack in 1 machine word (32 bits)
        //
        unsigned int time:31;   // event time
        Time_t type:1;          // type of timestamp: time/beats

public:
        // constructor
        EvTime() {
                // fill in the defaults
                time = 0;
                type = Tm_MIDI_Tick;
        };


        void SetTick(int t) {
                time = t;
                type = Tm_MIDI_Tick;
        };

        void SetTime(int t) {
                time = t;
                type = Tm_Abs_Time;
        };


        // should pass pointer to tempo track to perform time<->tick conversions
        int GetTick(void) {
                if (type == Tm_MIDI_Tick) {
                        return time;
                } else {
                        cerr << __FILE__ << " " << __LINE__ << " time to tick conversion not yet supported" << endl;
                        return 0;
                }
        };

        int GetTime(void) {
                if (type == Tm_Abs_Time) {
                        return time;
                } else {
                        cerr << __FILE__ << " " << __LINE__ << " time to tick conversion not yet supported" << endl;
                        return 0;
                }
        };
};


----------------------

> > >
> > >  MIDI events suck. MIDI should be a wrapper layer for those who have
> > > existing applications.
> > 
> > ???? I'm not sure we are talking about the same MIDI. If 'your' MIDI is the
> > low-level MIDI protocol, with all those byte sequences, and only limited to
> > a few standard events I totally agree with you. But (perhaps I was not clear
> > enough) the type of events I was thinking of are modeled after MIDI in the
> > spirit of "Make sound with volume aaa, pitch bbb, on channel ccc of
> > intrument (port) ddd", and "Change parameter aaa to value bbb on channel ccc
> > of instrument dddd". MIDI is limited in may ways because it consists of
> > short packets of 7 bit values. We do not have that limit for specifying an
> > event.
> 
>  Yep I was mainly talking about the byte sequence from the external
> devices. 
> I think as far as an aplication is concerned it wants to see a Voice. A
> voice
> should support all the standard midi functions if possible. The
> device/channel
> distinction is not needed at the highest level but is required when you
> create
> allocate a voice.

I think we have a subtile misunderstanding here.  I'm thinking of the MIDI
concept, I guess I did not communicate that clearly when using the
instrument/port in my example (stupid me). I should NOT have used the word
"instrument" when refering to a "port".

port    = (for MIDI) the interface to hook up the instruments (you know, the
          5 pin DIN connectors). When refering to an on-board syntheseiser,
          this can be seen as one (or perhaps more) ports. Each MIDI port
          supports 16 channels. Instead of port, we also refer to is as MIDI
          buss. 

channel = one of the 16 channels on a MIDI buss. There can be multiple notes
          playing concurrently on one channel.

In a physical MIDI setup, one can daisy chain the MIDI devices. Logically it
looks like all the daisy chained devices are parallel. One can setup the
devices (ie. synths) to 'listen' for a specific channel. Or one can have
synths that listen to many (or all) channels (most current synths do so).

A typical use is to use the MIDI channels to specify which 'instrument'
should play a certain events. Eg. the piano is set to channel 1, the bass
has channel 2, strings on channel 6, and drums on channel 10. 

If the synth is capable off (most do), we can sent multiple note on event to
a certain channel, to get the piano mentioned earlier to play some chords.

Small MIDI system: one or more synths connected to one single port. The
musician has only 16 channels for addressing events to the right 'sound'.
To overcome this limitation, more MIDI output ports can be added to a
system, where each port drives a separate MIDI buss. On a system with N
ports, the musician has access to N*16 different channels.


----------------------


> 
> 
> > Because I had my own home studio (with Steinberg Cubase as sequencer, an XG
> > synth module, a Kawai K1 synth, a Novation BassStation synth and a Boss RV70
> > Reverb) in mind while composing my proposal, I know that even with the
> > restricted 'bare bones' MIDI standard it is possible to access all
> > parameters in my equipment (and there are much more parameters to fiddle
> > with than can be found on a GUS or AWE...). When dealing with MIDI equipment
> > there are a few commonly used ways to give access to the sound
> > characteristics (I'm sure you are familiar with these):
> > 
> > 1) Program changes (only 0-127). To work around this restriction, synth
> >    vendors came up with two alternatives to enhance addressing: GS (Roland)
> >    and XG (Yamaha). Both standards use a controllor to select the bank the
> >    program will selected from.
> > 
> > 2) Other channel messages eg. pitch bed, channel aftertouch, polyphonic
> >    aftertouch.
> > 
> > 3) Control changes. (128 controllers, 7 bits). This is intended for real-time
> >    control of sound characteristics. Think of panning, volume, modulation,
> >    filter control: Frequency/Resonance (also known as 'Brightness' and
> >    'Harmonic Content' etc. To extend the numer of available controllers and
> >    range of the value (to 14 bits) Non Registered and Registered Parameter
> >    Numbers are used (RPN/NRPN). XG Makes heavy use of NRPN.
> > 
> > 4) System exclusive messages. Vendor/Synth specific. Used in almost all
> >    syntheseisers to allow access to every configurable or programmable bit.
> >    Allows even download of samples... These messages are mostly non portable
> >    and require 'computer programmer musicians' or special applications (synth
> >    editors, bank managers.
> > 
> > I think because of the huge installed base of MIDI equipment that should be
> > the first target to aim at, and use the methods shown above. For the synths
> > that are not 'stand-alone boxes that are hooked up with MIDI cables' like
> > the ones that are integrated on soundcards (OPL3, GUS, AWE, ...) I think the
> > best approach is to let the outside world (ie. the sequencer applications)
> > see these devices as 'just another synth'. All the specials and extras,
> > including programming capability should be wrapped in the standard available
> > control messages.
> > 
> > Advantages of this approach:
> > 
> > - From musicians's point of view an internal synth (eg. my GUS) doesn't look
> >   different from an external unit (both have there specific messages to
> >   massage the sound).
> > 
> > - Common interface for application programmer.
> > 
> > - If the Linux system is only used as patchbay, with MIDI->Internal Synth
> >   routing, one can use an external sequencer and make fully use of the
> >   internal unit. (Great PLUS, my GUS is basicly a neat sampler, but only
> >   usefull for playing MOD files... If I can drive it from Cubase on my
> >   other computer, I've just another synth. Same applies if one wants to hook
> >   up a MIDI keyboard, and want to play using internal sounds.
> 
>  I think this is a very difficult issue.
> 
>  The following is of the top of my head so it may be total rubish but I
> will throw
> it into the melting pot.

:-)

> 
>  I understand that midi allows you to control all the features of the
> synthesises.

Yep.
 
> The trouble is that to make the application software easy to use it
> needs to understand
> how to control your synth. For example some filter sweep may be
> controlled using 
> the non registered midi controls. This as you know requires sending a
> high order byte
> and a low order byte. Using cakewalk aprentice (it came with my card)
> implemeting a filter
> sweep is a long boring task because  . .  . well you can imagine why.

This can be made more user-friendly if the sequencer supports the NRPN 
controller as ONE event (instead of distinct messages). 

Usage can be made even easier if the is a standard for the controller
messages, and the application has predefined controller codes for the often
used ones. Steinberg Cubase support this, and lets you just pick a
controller from a list (also RPN/NRPN). 

But sadly there are only few things standarised in the not-so-perfect MIDI
world. To live with this, the user will always have to be given access to
the send 'just a controller' (and even sending the ugly high/low byte
sequences like you did with Cakewalk).

As the interface to the synths that are build into the soundcards can be
(and has to) defined by ourselfes, we can make them 'compatible'. For the
outboard MIDI equipment this is not possible. I think is a great win if this
standard is compatible with the majority of outside MIDI equipment. 



> Also say I want
> to play my song via another synth ? How does it know what to do with
> these AWE32 specifics ?
> 
>  So do we 
> 
> 1. Use device dependent messages in our representation and lose some
> possibilities
> of getting the desired effect on another synth. This scheme makes
> writting application
> software more difficult because it always needs to know about specifics.

This is indeed how 'current' MIDI world looks like. The GM standard was
invented to clean this mess.

> 
> 2. Try to define a generic synth and have a layer that translates the
> generic messages
> to specifics. In this case reading an existing file would need to
> translate the specific
> code to a generic. We would also need to be able to write out specifics
> from our generic.

Yep.

the translation from the 'generic synth + extensions' is done by the
specific device driver (=client to the sequencer core) for a specific
syntheseiser like AWE,GUS. 


> 
>  The generic layer would be based existing features so the mapping of
> most cases would be straight
> forward. They could live in the midi file as the non registered controls
> for a generic LINUX synthesiser.

Yep.

> 
> > > However, the design of good generic interface for the different devices
> > > is still quite tricky.
> > > If I have written a piece of music for my AWE32 using some of it's cool
> > > filter sweeps
> > > how will a GUS card play this ? One solution is to have adaptors e.g.
> > 
> > If it's a special AWE32 feature that itsn't supported by the GUS, it will
> > just play, but without the great effects. This same applies using 'normal'
> > syntheseisers. If a send data sequenced for my K1 to the XG module, it will
> > sound very different, and the XG lacks the great analog sound of the
> > BassStation, and my reverb has far more parameters than the reverb unit of
> > the XG synth etc. etc. etc. etc.
> > 
> > Bottom line: I don't think we should make life more complicated than it is,
> > and just accept the differences between the devices (otherwise one would
> > only but 1 synth and that's it; the reason for excistence of many devices
> > are their own strength and weaknesses.)
> > 
> > To make life a little more easy, we can try to standardize the internal
> > synths to GM and perhaps extend it with some of the XG features.
> 
>  Believe it or not I was trying to simplify things !!!!  FIlter sweeps
> live on
> many synths but use different protocols. 

That's the main reason why a we can't use ONE event for eg. filter sweeps.
Though we can define ONE event that works for most synths (and all the
onboard synths ofcourse) but not ALL external units.

I wish the multiple protocols to do the same didn't exist, but they do...

> I am saying we define our
> standard protocol
> for a sweep and each device driver can translate it to it's own form. In
> most cases
> this will simply involve a scaling and shift of the control value. Maybe

Yes, that's the whole idea....

> one
> of the existing protocols is general enough for us to use XG (I know
> zero about this)?

If our standard protocol is XG like, then we are compatible will ALL general
MIDI (GM) syntheseisers (is almost every box).

> 
> > > AWE32->GUS but this
> > > explodes as the number of devices increases. A generic device would need
> > > to know about main features
> > > of all cards. Then we can have AWE32->generic   generic->GUS
> > 
> > No. If you have a song composed for AWE32, it will only sound _exactly same_
> > on AWE32 and on nothing else (ehh, perhaps on a AWE33 or AWE64 of these are
> > enhanced versions of the former.)
> > 
> > I think a AWE32 can be seen as: "generic + AWE32 specials". The GUS can be
> > seend as "generic + GUS specials". A standalone XG module (eg. Yamaha MU90R)
> > is "generic + MU90R specials". If a AWE32 specific song is played to another
> > device, eg. GUS, the latter will not understand the specials and just
> > discard the extra data; the generic events will be played though
> 
>  and I see that the specials of the cards fall into common sets with
> different
> interfaces and suggest we attempt to make the music more portable. 

Yes, that's the idea. provide a uniform interface for the common 'specials'.


New:

The concept of 'plugging clients into the sequencer core' also allows one to
develop a client that does nothing more than translating events (conforming
to 'our' standard) to event that are specific to a certain MIDI synth. This
allows a user of for instance a 'BassStation' to program the filter by
sending the standarised event (client will map it to Bassstation specific).



----------------------
 
> 
> TO summarise.
> 
>  I think we agree about most stuff. The main issue for me is this
> question of
> how each synth appears to the application. From my point of view it
> would make
> the writing of my music software (cakewalk/cubase type stuff) easier if
> I could
> interface to a generic device other wise I have to have manage different
> representations
> etc. I am also saying that a generic representation would make things
> more portable allowing
> AWE32 songs to play on the GUS.
> 
>   cheers Paul.
>

Agreed, agreed, agreed!!!


And now I'll get me a beer, this typing makes me thursty.
cheers,
Frank.
 

========================---------------->
#define NAME    "Frank van de Pol"     
#define ADDRESS "mgr. Nelislaan 10"  
#define CITY    "4741 AB Hoeven"    
#define COUNTRY "The Netherlands"  
#define EMAIL   "F.K.W.van.de.Pol@inter.NL.net     -o)
                                                   /\\
Linux - Why use Windows, since there is a door?   _\_v

