Re: [PATCH virtio 0/8] virtio_pci_modern: allow parallel admin queue commands execution

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Mon, Jun 24, 2024 at 05:23:56PM CEST, mst@xxxxxxxxxx wrote:
>On Mon, Jun 24, 2024 at 05:10:36PM +0200, Jiri Pirko wrote:
>> Mon, Jun 24, 2024 at 01:23:01PM CEST, mst@xxxxxxxxxx wrote:
>> >On Mon, Jun 24, 2024 at 05:53:52PM +0800, Heng Qi wrote:
>> >> On Mon, 24 Jun 2024 11:04:43 +0200, Jiri Pirko <jiri@xxxxxxxxxxx> wrote:
>> >> > From: Jiri Pirko <jiri@xxxxxxxxxx>
>> >> > 
>> >> > Currently the admin queue command execution is serialized by a lock.
>> >> > This patchsets lifts this limitation allowing to execute admin queue
>> >> > commands in parallel. To do that, admin queue processing needs to be
>> >> > converted from polling to interrupt based completion.
>> >> > 
>> >> > Patches #1-#6 are preparations, making things a bit smoother as well.
>> >> > Patch #7 implements interrupt based completion for admin queue.
>> >> 
>> >> Hi, Jiri
>> >> 
>> >> Before this set, I pushed the cvq irq set [1], and the discussion focused on the
>> >> fact that the newly added irq vector may cause the IO queue to fall back to
>> >> shared interrupt mode.
>> >> But it is true that devices implemented according to the specification should
>> >> not encounter this problem. So what do you think?
>> >> 
>> >> [1] https://lore.kernel.org/all/20240619171708-mutt-send-email-mst@xxxxxxxxxx/
>> >
>> >It's true - this can cause guest to run out of vectors for a variety of
>> >reasons.
>> >
>> >First we have guest irqs - I am guessing avq could use IRQF_SHARED ?
>> >I am not sure why we don't allow IRQF_SHARED for the config
>> >interrupt though. So I think addressing this part can be deferred.
>> >
>> >Second, we might not have enough msix vectors on the device. Here sharing
>> >with e.g. cvq and further with config interrupt would make sense.
>> >
>> >Jiri do you think you can help Heng Qi hammer out a solution for cvq?
>> >I feel this will work will then benefit in a similar way,
>> >and having us poll aggressively for cvq but not admin commands
>> >does not make much sense, right?
>> 
>> 
>> How about to enhance existing fallback in vp_find_vqs() to something
>> like this:
>> 
>>         /* Try MSI-X with one vector per queue. */
>>         err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, ONE_FOR_EACH_VQ, ctx, desc);
>>         if (!err)
>>                 return 0;
>>         /* Try MSI-X with one vector per data queue, share one for the
>>          * rest of the queues and config. */
>>         err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, ONE_FOR_EACH_DATA_VQ_ONE_FOR_REST, ctx, desc);
>>         if (!err)
>>                 return 0;
>>         /* Fallback: MSI-X with one vector for config, one shared for queues. */
>>         err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, ONE_DATA_VQS_ONE_FOR_REST, ctx, desc);
>>         if (!err)
>>                 return 0;
>> 
>> I can implement this for admin queue, control queue can benefit from
>> this eventually.
>> 
>
>OKay superficially except in attempt 3 I would still combine
>adminq with config, for consistency.

Yeah, I thought about that when I wrote that. Makes sense.
I'm on it.


>
>> 
>> >
>> >> > Patch #8 finally removes the admin queue serialization lock.
>> >> > 
>> >> > Jiri Pirko (8):
>> >> >   virtio_pci: push out single vq find code to vp_find_one_vq_msix()
>> >> >   virtio_pci_modern: treat vp_dev->admin_vq.info.vq pointer as static
>> >> >   virtio: push out code to vp_avq_index()
>> >> >   virtio: create admin queues alongside other virtqueues
>> >> >   virtio_pci_modern: create admin queue of queried size
>> >> >   virtio_pci_modern: pass cmd as an identification token
>> >> >   virtio_pci_modern: use completion instead of busy loop to wait on
>> >> >     admin cmd result
>> >> >   virtio_pci_modern: remove admin queue serialization lock
>> >> > 
>> >> >  drivers/virtio/virtio.c            |  28 +----
>> >> >  drivers/virtio/virtio_pci_common.c | 109 ++++++++++++++------
>> >> >  drivers/virtio/virtio_pci_common.h |   9 +-
>> >> >  drivers/virtio/virtio_pci_modern.c | 160 ++++++++++++-----------------
>> >> >  include/linux/virtio.h             |   2 +
>> >> >  include/linux/virtio_config.h      |   2 -
>> >> >  6 files changed, 150 insertions(+), 160 deletions(-)
>> >> > 
>> >> > -- 
>> >> > 2.45.1
>> >> > 
>> >> > 
>> >
>




[Index of Archives]     [KVM Development]     [Libvirt Development]     [Libvirt Users]     [CentOS Virtualization]     [Netdev]     [Ethernet Bridging]     [Linux Wireless]     [Kernel Newbies]     [Security]     [Linux for Hams]     [Netfilter]     [Bugtraq]     [Yosemite Forum]     [MIPS Linux]     [ARM Linux]     [Linux RAID]     [Linux Admin]     [Samba]

  Powered by Linux