From alsa-devel-owner@alsa.jcu.cz  Thu May  7 00:40:53 1998
Received: from obelix.fvdpol.inter.nl.net (root@bd99-8.Breda.NL.net [193.79.246.233])
	by marvin.jcu.cz (8.8.8/8.8.8) with ESMTP id AAA30877
	for <alsa-devel@jcu.cz>; Thu, 7 May 1998 00:40:23 +0200
Received: (from frank@localhost) by obelix.fvdpol.inter.nl.net (8.8.7/8.7.3) id XAA26909 for alsa-devel@jcu.cz; Mon, 4 May 1998 23:50:48 +0200
From: Frank van de Pol <F.K.W.van.de.Pol@Inter.NL.net>
Message-Id: <199805042150.XAA26909@obelix.fvdpol.inter.nl.net>
Subject: Re: New Sequencer core
To: alsa-devel@jcu.cz
Date: Mon, 4 May 1998 23:50:47 +0200 (MET DST)
In-Reply-To: <354CB5B9.53DA6144@arrakis.es> from "Antonio Larrosa" at May 3, 98 06:21:46 pm
Content-Type: text
Reply-To: alsa-devel@alsa.jcu.cz
Sender: alsa-devel-owner@alsa.jcu.cz
Precedence: list

Antonio Larrosa wrote:
> 
> I wrote:
> > 
> > I also had some thoughts in this issue. When implementing a priority queue,
> > we need to be able to compare time stamps of events. If there is a constant
> > tempo, that can easily be done. But when a tempo is changed, the number of
> > clock ticks (ppq) per second changes, and events in the queue will not have
> > correct ordering anymore. To avoid this problem, only one type of timestamp
> > is needed. We have basicly two options to choose from:
> > 
> > 1) real/clock-time, expressed in seconds (or fractions of it of course,
> >    eg. ms or us.
> > 
> > 2) songposition, expressed in clock ticks, which are related to the tempo
> >   the song is playing. for an internal resolution of eg. 1920 PPQ (parts per
> >   quarter noter), and a tempo of 135 BPM, there are 1920*135=259200 clock ticks
> >   per minute = 4320 ticks per second.
> > 
> > Because of the specific character of music (ie. it has tempo, groove etc.)
> > the second one is prefered. Conversion between these two types is trivial
> > once the currently active tempo and timestamp (in both units!) of last tempo
> > change is known.
> 
> I'm not too sure of this being a good solution . A ms. is a time unit that
> never changes (not aplying theorical physics :-)), but controlling clock ticks
> is very tricky when there are several time changes . 
> Suppose this situation :
> 
> Music starts playing, you place a change tempo event at tick 5000 . Before it
> is processed, you place another change tempo event at tick 10000 .
> Now, what is tick 10000 ? , is the 10000 ticks ms. calculated using the
> current tempo ?, or you are going to run over the list of queued events to
> take care of that tempo change that is at tick 5000 ?

When using the songposition notation, a tick is and keeps being a fraction a
a bar; changing the tempo while playing, will just result in a different
number of ticks per second. Events (eg. hihat pattern) that is queued with
the 'old' tempo will then follow the new tempo while staying in groove. The
tick->second calculation has only to be performed for the event that is to
be dequeued. If a time notation (in seconds) was used instead, we indeed
should have to run over the list of queued events.

If multiple tempo changes are queued, like in your example, that will work
out fine as I'll illustate later.

> 
> That's why I think that time should be expressed in ms. or us. 

Both methods have their advantages and disadvantages. For note events the
tempo method is really the best (less trouble for tempo changes). But for
events that we want to fire at an exact time stamp (eg. synced to sample
playback!), we better have time stamps in seconds (or ms, or us). If there
are no (zero, 0) tempo changes, it even doesn't matter wich method to use!

To overcome this compromise, we can use two parallel queues, one for the
real-time (seconds), and one for the song position (ticks). 

Regarding tempo calculations:

The internal clock always runs at a constant time (well, at least that's
what we assume, every crystal deviates a few PPM :-). To illustate what
happens with tempo changes that are enqueued (like in your example) I'll try
to give an example.

Initial tempo 120 BPM, sequencer resolution 384 PPQ, starting time and song
position start at 0, assume 4/4 signiture.

Little piece of terminology: the 384 PPQ means there are 384 parts ('ticks')
per quarter note, the 120 BPM means that there are 120 beats (=quater notes)
per minute. The 4/4 signiture only indicates we have 4 quarter notes (/4)
per bar.

Assume we want to have a note every beat for a whole song, then the
sequencer will have to enqueue the note with a 'songpos' of 0, 384, 768,
1152, 1536 etc. etc. (n * 386). Note that this is independent of tempo, the
note (say a bassdrum) is always triggered at the same interval within a bar.

I assumed a initial tempo of 120 BPM, so there are 120/60 = 2 beats per
second, or (120/60)*384 = 768 ticks per second. or 1/((120/60)*384) = 1.3E-3
seconds per tick.

If the current clock time is 5.2 seconds since start of the song, and the
tempo didn't change, the event with a timestamp of 5.2*768 = 3993.6 (or
earlier) has to be played. 

If a tempo change was scheduled at say time stamp 6144 (the first beat of
the 4th bar, perhaps the end of an intro), and sets the new tempo to 145
BPM, the song will keep on playing with the old (last known) tempo of 120
BPM until it reaches timestamp 6144 (at t=8.0 s). So far so good. 
The tempo change will now be fetched from the queue and update the tempo
'variables'. 

store:
time of tempo change	= 8.0 s
songpos of tempo change	= 6144
new tempo = 145.

(our bass drum is still pumping every beat!)

With this new tempo we have (145/60)*384 = 928 ticks per second. If the
current clock time is now at 11 seconds since start of song, the event will
have a timestamp of [here comes the trick!]: 6144+((11-8.0)*928) = 8928.
Calulating the songposion as an offset from last tempo change makes sure we
always get the right time & timestamp. 

The first beat of bar 12, with a timestamp of (12*4*384)=18432, has a
schedule time of 8.0+((18432-6144)/928) = 21.24 seconds.

More (and/or often) tempo changes do not matter for this scheme; if we keep
the time and songposition of last tempo change, and the new tempo, we can
always (very easy) calculate the songposition from clock time and clock time
from songposition. 

The internal clock can of course just keep on ticking at the same interval,
10ms for the standard system clock, or faster if another timer source is
used.

Is there perhaps some subtile issue I overlooked or misunderstood from your
posting?

> 
> [cut]
> 
> > To make life a little more easy, we can try to standardize the internal
> > synths to GM and perhaps extend it with some of the XG features.
> [cut] 
> > I think a AWE32 can be seen as: "generic + AWE32 specials". The GUS can be
> > seend as "generic + GUS specials". A standalone XG module (eg. Yamaha MU90R)
> > is "generic + MU90R specials". If a AWE32 specific song is played to another
> > device, eg. GUS, the latter will not understand the specials and just
> > discard the extra data; the generic events will be played though
> 
> I agree completely in both points.
> 
> However I have to note something. In the initial message  (sorry, not quoted),
> you told something about multiple clients being able to open the same
> device r/w, so that many applications can use midi output at the same time.

Perhaps some more explanation is needed to clarify my idea (which is
completely different from OSS's model):

"Client" in my proposal is just _anything_ that communicates with the
sequencer core. The sequencer core is nothing more than a foundation that
does nothing but _queueing_ and _routing messages_ between clients. These
clients can either reside in userland or kernel land, and can be
applications like sequencers, midi players etc., or can be device drivers
that play sounds (MIDI, synths, soft synths) or have inputs (MIDI keyboards,
drums etc.), or simply 'massage' the data and act as filters, mappers. 

In a typical minimal system that is simply playing a MIDI file, there are
always a few clients:

- the MIDI playback application (userland of course). This only thing the
  'user' behind the machine will recognise as a client.

- a client withing the sequencer core, that only handles the tempo changes. 

- a client that resides in the kernel, perhaps in the driver for the MIDI
  interface (GUS, MPU401, ...)

MIDI players sends events into the queue, with a destination address set to
the MIDI port. The sequencer core will do the scheduling, and deliver the
event to the MIDI port (or syntheseiser) when the time is there. This is
basicly the same as you can get with OSS. 

The difference with OSS is that there is a router after the queue, and that
a priority queue is used instead of a simple FIFO. Clients can put put in
chains if you want. 

In the above simple example for a MIDI player, one can add a additional
client that allows for instance to change the pitch of certain note events
(only specific channels). All this imaginary client does is simply shift the
pitch of the received data by 2 semi-tones, and send it to the MIDI output.
If now the MIDI player does not send it's events directly to the MIDI outpu,
but sends it to this 'pitchshift' client; we get to hear the same song, now
with a different pitch. Okay you'll think; whats the use for this pitch
shifter? Not much I think, but it illustates one big strenght of this
concept:

The Sequencing and event handling is build upon a few components and not on
one big monolithic "we want it to do everything" driver. There is NO NEED to
implement MIDI thru, filtering etc. in the sequencer core. If one wants to
have that functionality, just add a client. If many users (or even all) want
to have a certain piece of functionality, we can event include it as a
seperate client (either user-land application or kernel module) separate
with the driver.

The presencee of a priority queue instead of a FIFO allow us to add new
events to be played now or shortly to an queue with events that are already
queued for the future. No need to mess with the out-of-band ioctl's.
      
> 
> I think this is not desirable. No, don't get me wrong, of course many
> applications should be able to open midi devices, but I think that it would
> be better done with a midi server than in kernel mode .

Reason for me to want to have the scheduling in the kernel is because that's
the only place where I can expect events to be delivered in time. Playback
(and recording) can here be triggered by any timer or other (MIDI rx)
interrupt. That's the best timing you can get. Why does timing suck on
microsoft windows, when compared to the good old Atari? That's because the
Atari can have everything interrupt driven, and windows can't.

The concept is not designed (be does allow) to have multiple application
playing with the same devices. The basic idea is to make a framework that
can be used build sequencer systems. This can range from low-end midi or MOD
players to high-end (possibly distributed over many nodes) sequencer and
real-time computer synthesis systems. The framework should not be
restrictive and allow to grow in the future (That's why it is not a n OSS
/dev/music emulator.).

> Everything that could be outside the kernel is better out than in . Perhaps
> we could distribute the midi server with the driver, but not inside it .

It's up to the client developer where to place his or her client: in or
outside the kernel. I can imagine we could like to have a soft synth.
Instead of going though the trouble of developing one in kernel, we can
start off with running user-land timidity, and let our MIDI playing
applications send their data to the 'timidity client' instead of the 'GUS
MIDI' client. For the player it doesn't make a difference (only another
destination), and whooom, we have a soft-synth.

> 
> Another argument on this is that the driver should be kept the simpler we
> (Jaroslav) can make it :-)

Fully agreed!!! That's why the sequencer core only has a few queue's and a
event router. 

- All the ugly hardware specific details that make life complicated can go
  into the parts where they belong: the device specific drivers. 

- No need to keep adding new functionality (like midi thru etc.) to the 
  sequencer because the sequencer won't do more than sequencing and routing
  events from one client to another. All the bells and whistles ("I want to
  have it make coffee") can put in clients.

- OSS compatibility can be guaranteed by creating a client with /dev/music
  and /dev/sequencer interface that simply maps the OSS events to ALSA events.
  And perhaps even a client that calls the OSS to perform/play ALSA events
  (use OSS as low-level I/O).

- The concept can be quite clean and easy so that is would not be a problem
  to implement it. (Done by me or other volunteers, then Jaroslav and others
  can keep concentrating on the low-level hardware driver where the ALSA
  still needs a bunch off.)


> 
> Greetings,
> 
> -- 
> Antonio Larrosa Jimenez
> antlarr@arrakis.es

Thanks for you response, this means that at least some people take the
trouble to go though that long story of me. I can understand that it's hard
to understand my ideas because they are not in the line of the OSS approach
to sequencing all Linux developers are used to, and have some fundamentally
different approach. My story is also not complete, I have more 'blurry
ideas' on this subject, and could talk for hours on this idea (I did!), but
putting it down in a way that understands is not without trouble (not an
easy task, to translate Brain->Email :-) Perhaps reading other (recent)
postings on this subject might help.

Let me know what you think, and what parts are not clear. 

Regards,
Frank.

BTW: for your kmid application (your posting 2 weeks ago whas for me the
trigger to put my idea down; I have this in my head for a long time) you
said you wanted to have real-time volume changes.

That's very easy with this new architecture (ALSA = Advanced Linux Sequencing
Architecture):

- You can simply enqueue volume change events (controller data) with a
  timestamp of _now_.

- Or if you like to go into more trouble, you can add a client to intercept
  the events, and change the velocity, or even mute it before passing it to
  the destination. (better put that client in kernel to get best timing. A
  RT scheduled client might also give good results.)

Regards,
Frank.


========================---------------->
#define NAME    "Frank van de Pol"     
#define ADDRESS "mgr. Nelislaan 10"  
#define CITY    "4741 AB Hoeven"    
#define COUNTRY "The Netherlands"  
#define EMAIL   "F.K.W.van.de.Pol@inter.NL.net     -o)
                                                   /\\
Linux - Why use Windows, since there is a door?   _\_v

