Mon, Jun 24, 2024 at 05:23:56PM CEST, mst@xxxxxxxxxx wrote: >On Mon, Jun 24, 2024 at 05:10:36PM +0200, Jiri Pirko wrote: >> Mon, Jun 24, 2024 at 01:23:01PM CEST, mst@xxxxxxxxxx wrote: >> >On Mon, Jun 24, 2024 at 05:53:52PM +0800, Heng Qi wrote: >> >> On Mon, 24 Jun 2024 11:04:43 +0200, Jiri Pirko <jiri@xxxxxxxxxxx> wrote: >> >> > From: Jiri Pirko <jiri@xxxxxxxxxx> >> >> > >> >> > Currently the admin queue command execution is serialized by a lock. >> >> > This patchsets lifts this limitation allowing to execute admin queue >> >> > commands in parallel. To do that, admin queue processing needs to be >> >> > converted from polling to interrupt based completion. >> >> > >> >> > Patches #1-#6 are preparations, making things a bit smoother as well. >> >> > Patch #7 implements interrupt based completion for admin queue. >> >> >> >> Hi, Jiri >> >> >> >> Before this set, I pushed the cvq irq set [1], and the discussion focused on the >> >> fact that the newly added irq vector may cause the IO queue to fall back to >> >> shared interrupt mode. >> >> But it is true that devices implemented according to the specification should >> >> not encounter this problem. So what do you think? >> >> >> >> [1] https://lore.kernel.org/all/20240619171708-mutt-send-email-mst@xxxxxxxxxx/ >> > >> >It's true - this can cause guest to run out of vectors for a variety of >> >reasons. >> > >> >First we have guest irqs - I am guessing avq could use IRQF_SHARED ? >> >I am not sure why we don't allow IRQF_SHARED for the config >> >interrupt though. So I think addressing this part can be deferred. >> > >> >Second, we might not have enough msix vectors on the device. Here sharing >> >with e.g. cvq and further with config interrupt would make sense. >> > >> >Jiri do you think you can help Heng Qi hammer out a solution for cvq? >> >I feel this will work will then benefit in a similar way, >> >and having us poll aggressively for cvq but not admin commands >> >does not make much sense, right? >> >> >> How about to enhance existing fallback in vp_find_vqs() to something >> like this: >> >> /* Try MSI-X with one vector per queue. */ >> err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, ONE_FOR_EACH_VQ, ctx, desc); >> if (!err) >> return 0; >> /* Try MSI-X with one vector per data queue, share one for the >> * rest of the queues and config. */ >> err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, ONE_FOR_EACH_DATA_VQ_ONE_FOR_REST, ctx, desc); >> if (!err) >> return 0; >> /* Fallback: MSI-X with one vector for config, one shared for queues. */ >> err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names, ONE_DATA_VQS_ONE_FOR_REST, ctx, desc); >> if (!err) >> return 0; >> >> I can implement this for admin queue, control queue can benefit from >> this eventually. >> > >OKay superficially except in attempt 3 I would still combine >adminq with config, for consistency. Yeah, I thought about that when I wrote that. Makes sense. I'm on it. > >> >> > >> >> > Patch #8 finally removes the admin queue serialization lock. >> >> > >> >> > Jiri Pirko (8): >> >> > virtio_pci: push out single vq find code to vp_find_one_vq_msix() >> >> > virtio_pci_modern: treat vp_dev->admin_vq.info.vq pointer as static >> >> > virtio: push out code to vp_avq_index() >> >> > virtio: create admin queues alongside other virtqueues >> >> > virtio_pci_modern: create admin queue of queried size >> >> > virtio_pci_modern: pass cmd as an identification token >> >> > virtio_pci_modern: use completion instead of busy loop to wait on >> >> > admin cmd result >> >> > virtio_pci_modern: remove admin queue serialization lock >> >> > >> >> > drivers/virtio/virtio.c | 28 +---- >> >> > drivers/virtio/virtio_pci_common.c | 109 ++++++++++++++------ >> >> > drivers/virtio/virtio_pci_common.h | 9 +- >> >> > drivers/virtio/virtio_pci_modern.c | 160 ++++++++++++----------------- >> >> > include/linux/virtio.h | 2 + >> >> > include/linux/virtio_config.h | 2 - >> >> > 6 files changed, 150 insertions(+), 160 deletions(-) >> >> > >> >> > -- >> >> > 2.45.1 >> >> > >> >> > >> > >
