Hi, On Fri, Mar 26, 2021 at 08:55:38AM +0100, Takashi Iwai wrote: > On Fri, 26 Mar 2021 05:46:15 +0100, Takashi Sakamoto wrote: > > On Wed, Mar 24, 2021 at 04:57:31PM +0100, David Henningsson wrote: > > > > However, the timing jitter of IRQ handler invocation is issued in this > > > > case, as well as PCM interface, even if the data rate of MIDI physical > > > > layer is quite low nowadays (31.25 Kbit / sec ~= 3906.25 byte / sec). > > > > As long as I experienced, in actual running Linux system, the invocation > > > > of IRQ handler has no guarantee for timing jitter, mainly due to CPU level > > > > IRQ mask (like spin_lock). Therefore the interval of each invocation is not > > > > so precise to decide event timestamp, at least for time slot comes from > > > > MIDI physical layer. > > > > > > > > Nevertheless, I think your idea is enough interesting, with conditions to > > > > deliver information from driver (or driver developer) to applications > > > > (ALSA Sequencer or userspace applications). Even if we have some > > > > limitation and restriction to precise timestamp, it's worth to work for > > > > it. It seems to be required that improvements at interface level and > > > > documentation about how to use the frame timestamp you implemented. > > > > > > Right, so first, I believe the most common way to transport midi these days > > > is through USB, where the 31.25 Kbit/sec limit does not apply: instead we > > > have a granularity of 1/8 ms but many messages can fit in each one. So that > > > limit is - for many if not most cases - gone. > > > > > > Second; you're probably right in that there is still some jitter w r t when > > > the IRQ fires. That is regrettable, but the earlier we get that timestamp > > > the better, so a timestamp in IRQ context would at least be better than in a > > > workqueue that fires after the IRQ, or in userspace that possibly has > > > scheduling delays. > > > > > > Btw, I checked the "struct urb" and there was no timestamp in there, so I > > > don't know how to get a better timestamp than in snd_rawmidi_receive. But > > > maybe other interfaces (PCI, Firewire etc) offers something better. > > > > Hm. Regardless of type of hardware; e.g. OHCI/EHCI/xHCI, 1394 OHCI, or > > PCI-e extension card, for software to process sampled data, it's always > > issue that the jitter between triggering IRQ (hardware side) and invoking > > IRQ handler (processor side). As an actual example, let me share my > > experience in 1394 OHCI. > > > > > > 1394 OHCI controller is governed by 24.576 Mhz clock (or double depending > > on vendors). In IEEE 1394, ishcornous service is 8,000 times per second. > > 1394 OHCI specification allows software to schedule hardware IRQ per > > isochronous cycle. > > > > Once ALSA firewire stack is programmed to schedule the hardwar IRQ evenry > > 16 isochronous cycle. The corresponding IRQ handler is expected to invoke > > every 2 milli second. As long as I tested in usual desktop environment[2], > > the jitter is between 150 micro second and 4.7 milli second. In the worst > > case, it's 6.0 milli seconds. The above is also the same wnen using > > 'threadirqs'. > > > > When using 'isolcpus' and arranging 'smp_affinity' to reduce load from one > > of processor core to invoke the IRQ handler, the interval is 2 milli > > second with +- several nano seconds, therefore the 1394 OHCI controller > > can trigger hardware IRQ so precise. The jitter comes from processor side. > > I think many running contexts on the same processor core masks IRQ so > > often and the jitter is not deterministic. > > > > Here, what I'd like to tell you is that we can not handle the system time > > as is as timestamp of received MIDI messages, as long as relying on IRQ > > context. We need some arrangements to construct better timestamp with some > > compensations. In this point, the 3rd version of patch is not enough[3], > > in my opinion. > > > > My intension is not to discourage you, but it's better to have more care. > > As one of the care, I think we can use extension of > > 'struct snd_rawmidi_status' to deliver some useful information to ALSA > > rawmidi applications, or including other parameters to the frame structure. > > But I don't have ideas about what information should be delivered, > > sorry... > > Well, the question is how much accuracy is wanted, and it's relatively > low for MIDI -- at least v1, which was defined many decades ago for a > slow serial line. > > That said, the patch set was designed for providing the best-effort > timestamping in software, and that's supposed to be enough for normal > use cases. Indeed. I think it the best-effort in software side, but with less care of hardware matters. Although I have no strong objection to the idea of the patch itself, the point of my insistent is _not_ how much accuracy is preferable. When I imagine to write actual program as an application of rawmidi with the frame structure, I have a concern about how to use the timestamp. At present, it's just a record of invocation of any IRQ context. When using it for timestamp of MIDI message, the application needs supplement information for compensation for any jitter and delay, however nothing is provided. At present, we seem to fail setting any incentive for users to use the new functionality, in my opinion. Such functionality is going to be obsoleted sooner or later, like 'struct snd_rawmidi_status.tstamp' (I guess no one could have implemented it for the original purpose, then no one is going to use it). That's my concern. > If there is any device that is with the hardware timestamping, in > theory, we could provide the similar data stream in the same format > but maybe with a different timestamp type. I seem to fail getting what you mean, but if I'm allowed to mention about legacy devices, I'll report the case of IEC 61883-1/6 in IEEE 1394 with 1394 OHCI. 1394 OHCI allows software to handle isochronous packet payload with cycle time. We can compute the timestamp independent on current system time in IRQ handler with resolution by 125 micro second. Below drivers can adapt to it: * snd-bebob * snd-fireworks * snd-oxfw * snd-dice * snd-firewire-digi00x (mutated version of protocol) * snd-firewire-motu (mutated version of protocol) But for the case below MIDI messages are on asynchronous packet and need to current system time in invoked IRQ handler: * snd-tascam * snd-fireface > But actually I'd like to see some measurement how much we can improve > the timestamp accuracy by shifting the post office. This may show > interesting numbers. > > Also, one thing to be more considered is the support for MIDI v2 in > future. I haven't seen any development so far (and no device > available around), so I cannot comment on this much more, but it'd be > worth to take a quick look before defining the solid API/ABI. Regards Takashi Sakamoto
