Re: [PATCH v30 00/30] Introduce QC USB SND audio offloading support

[Takashi Iwai <tiwai@xxxxxxx>]

On Tue, 10 Dec 2024 01:59:10 +0100,
Wesley Cheng wrote:
> 
> On 12/5/2024 4:53 PM, Wesley Cheng wrote:
> > On 12/4/2024 2:49 PM, Pierre-Louis Bossart wrote:
> >>>
> >>>>> UAOL is one of our priorities right now and some (e.g.: me) prefer to not pollute their mind with another approaches until what they have in mind is crystalized. In short, I'd vote for a approach where USB device has a ASoC representative in sound/soc/codecs/ just like it is the case for HDAudio. Either that or at least a ASoC-component representative, a dependency for UAOL-capable card to enumerate.
> >>>> The main difference is that we donʼt want the USB audio *control* part to be seen in two places. The only requirement is to stream data with an alternate optimized path, but all the volume control and whatnot is supposed to be done using the regular usb-audio card. It would be complete chaos for userspace if the same volume can be represented differently.
> >>>> The comparison with HDaudio is not quite right either. In the case of HDaudio, itʼs an all-or-nothing solution. The external device is controlled by one entity, either legacy or ASoC based. That choice is made at driver probe time. In the case of USB, the application needs to have the choice of using either the legacy path, or the optimized path that goes through a DSP. I think the last thing you want given this context is to make the USB audio device an ASoC codec.
> >>>> I find it rather interesting that this architectural feedback comes at the v30, itʼs unfair to Wesley really...
> >>> Hi Pierre,
> >>>
> >>> Obviously I'm late. After scanning the history of this one, indeed it's been a while since v1 has been sent. And thus I posted no NACKs. At the same time if I am to choose between: provide feedback vs provide no-feedback, I'd rather choose the former even if I'm to be ignored/overridden by a subsystem maintainer.
> >>>
> >>> The subsystem maintainers also hold the last word, and I have no problem with them merging the patches if they believe its existing shape is good-enough. For example, my team could follow up this implementation next year with a patchset expanding/updating the functionality. I see this as a viable option.
> >> Thatʼs what we had in mind before I left Intel. The interfaces seen by userspace are PCM devices and kcontrols, it doesnʼt matter too much if there is one card, two cards, and if the implementation relies on an ASoC codec, a library or something else. 
> >> The bulk of the work is to enable the USB offload from top to bottom, by changing PipeWire/CRAS/HAL to select the new optimized path when available and deal with plug/unplug events.
> >> Improvements at the kernel level can be done later if required. Itʼs hard to argue that the proposal in this series is fundamentally broken, but as usual itʼs likely that some requirements are missing or not known yet. The same thing happened with compressed offload, none one thought about gapless playback until Android made it a requirement. Maybe what weʼd need is a ʽprotocol versionʼ for USB offload so that changes can be tracked and handled?
> >
> > Thanks for chiming in, Pierre.  So for now, with the next revision I have prepared, I'm currently adding:
> >
> > 1.  Some improvements to xHCI sideband to account for core sequences that need to be notified to the offload driver, ie transfer ring free
> >
> > 2.  Moved the USB SND offload mixer driver into the QC vendor module for now, as instructed by Takashi:
> >
> > https://lore.kernel.org/linux-usb/87cyiiaxpc.wl-tiwai@xxxxxxx/
> >
> > 3.  Added separate kcontrols for fetching mapped PCM device and card indexes (versus one that returns a card and PCM device pair [array])
> >
> > 4.  Removed some jack controls (enable/disable) from soc-usb
> >
> > 5.  Updated documentation for #3
> >
> >
> > Those are the major changes that will come in the next revision.  I'm just trying to figure out who/where the "protocol version" should be checked if we decided to add it.  (or if we need to check for it anywhere...)  From the userspace perspective, it should be agnostic to how we've implemented offloading from the kernel, and I don't see any major shifts in how userspace implements things even if we make improvements from kernel.
> 
> 
> Hi Takashi,
> 
> Could you possibly help share some direction on what you think of the current design, and if you think its detrimental that we make modifications mentioned by Cezary?  I have the next revision ready for review, but I wanted to get a better sense on the likeliness of this landing upstream w/o the major modifications.

Honestly speaking, I have no big preference about that design
question.  The most important thing is rather what's visible change to
users.  An advantage of the current design (sort of add-on to the
existing USB-audio driver) is that it's merely a few card controls
that are added and visible, and the rest is just as of now.  The
remaining design issue (two cards or single card) is rather
kernel-internal, and has nothing to do with users.  So I'm fine with
the current design.

OTOH, if we follow the pattern of HD-audio, at least there will be
more preliminary changes in USB-audio driver side like we've done for
HD-audio.  That is, make most of USB-audio code to be usable as (a
kind of) library code.  It's more work, but certainly doable.  And if
that can be achieved and there other similar use cases of this stuff
not only from Qualcomm, it might make sense to go in that way, too.
That said, it's rather a question about what's extended in future.
If Intel will need / want to move on that direction, too, that's a
good reason to reconsider the basic design.


thanks,

Takashi