Mon, Jun 24, 2024 at 03:55:53PM CEST, mst@xxxxxxxxxx wrote: >On Mon, Jun 24, 2024 at 03:46:19PM +0200, Jiri Pirko wrote: >> Mon, Jun 24, 2024 at 01:23:01PM CEST, mst@xxxxxxxxxx wrote: >> >On Mon, Jun 24, 2024 at 05:53:52PM +0800, Heng Qi wrote: >> >> On Mon, 24 Jun 2024 11:04:43 +0200, Jiri Pirko <jiri@xxxxxxxxxxx> wrote: >> >> > From: Jiri Pirko <jiri@xxxxxxxxxx> >> >> > >> >> > Currently the admin queue command execution is serialized by a lock. >> >> > This patchsets lifts this limitation allowing to execute admin queue >> >> > commands in parallel. To do that, admin queue processing needs to be >> >> > converted from polling to interrupt based completion. >> >> > >> >> > Patches #1-#6 are preparations, making things a bit smoother as well. >> >> > Patch #7 implements interrupt based completion for admin queue. >> >> >> >> Hi, Jiri >> >> >> >> Before this set, I pushed the cvq irq set [1], and the discussion focused on the >> >> fact that the newly added irq vector may cause the IO queue to fall back to >> >> shared interrupt mode. >> >> But it is true that devices implemented according to the specification should >> >> not encounter this problem. So what do you think? >> >> Wait. Please note that admin queue is only created and used by PF virtio >> device. And most probably, this is on hypervisor managing the VFs that >> are passed to guest VMs. These VFs does not have admin queue. >> >> Therefore, this is hardly comparable to control vq. > > >Well Parav recently posted patches adding admin queue >to VFs, with new "self" group type. Right, but even so, when device implementation decides to implement and enable admin queue, it should also make sure to provide correct amount of vectors. My point is, there should not be any breakage in user expectation, or am I missing something? > > >> >> >> >> >> [1] https://lore.kernel.org/all/20240619171708-mutt-send-email-mst@xxxxxxxxxx/ >> > >> >It's true - this can cause guest to run out of vectors for a variety of >> >reasons. >> > >> >First we have guest irqs - I am guessing avq could use IRQF_SHARED ? >> >> There is no avq in quest, under normal circumstances. Unless for some >> reason somebody passes trough virtio PF into guest. > > >At the moment, but this will change soon. > > >> >> >I am not sure why we don't allow IRQF_SHARED for the config >> >interrupt though. So I think addressing this part can be deferred. >> > >> >Second, we might not have enough msix vectors on the device. Here sharing >> >with e.g. cvq and further with config interrupt would make sense. >> >> For cvq irq vector, I believe that sharing with config irq makes sense. >> Even for admin queue maybe. But again, admin queue is on PF. I don't >> think this is a real concern. >> >> >> > >> >Jiri do you think you can help Heng Qi hammer out a solution for cvq? >> >I feel this will work will then benefit in a similar way, >> >and having us poll aggressively for cvq but not admin commands >> >does not make much sense, right? >> > >> >> > Patch #8 finally removes the admin queue serialization lock. >> >> > >> >> > Jiri Pirko (8): >> >> > virtio_pci: push out single vq find code to vp_find_one_vq_msix() >> >> > virtio_pci_modern: treat vp_dev->admin_vq.info.vq pointer as static >> >> > virtio: push out code to vp_avq_index() >> >> > virtio: create admin queues alongside other virtqueues >> >> > virtio_pci_modern: create admin queue of queried size >> >> > virtio_pci_modern: pass cmd as an identification token >> >> > virtio_pci_modern: use completion instead of busy loop to wait on >> >> > admin cmd result >> >> > virtio_pci_modern: remove admin queue serialization lock >> >> > >> >> > drivers/virtio/virtio.c | 28 +---- >> >> > drivers/virtio/virtio_pci_common.c | 109 ++++++++++++++------ >> >> > drivers/virtio/virtio_pci_common.h | 9 +- >> >> > drivers/virtio/virtio_pci_modern.c | 160 ++++++++++++----------------- >> >> > include/linux/virtio.h | 2 + >> >> > include/linux/virtio_config.h | 2 - >> >> > 6 files changed, 150 insertions(+), 160 deletions(-) >> >> > >> >> > -- >> >> > 2.45.1 >> >> > >> >> > >> > >
