On Mon, Jun 24, 2024 at 03:46:19PM +0200, Jiri Pirko wrote: > Mon, Jun 24, 2024 at 01:23:01PM CEST, mst@xxxxxxxxxx wrote: > >On Mon, Jun 24, 2024 at 05:53:52PM +0800, Heng Qi wrote: > >> On Mon, 24 Jun 2024 11:04:43 +0200, Jiri Pirko <jiri@xxxxxxxxxxx> wrote: > >> > From: Jiri Pirko <jiri@xxxxxxxxxx> > >> > > >> > Currently the admin queue command execution is serialized by a lock. > >> > This patchsets lifts this limitation allowing to execute admin queue > >> > commands in parallel. To do that, admin queue processing needs to be > >> > converted from polling to interrupt based completion. > >> > > >> > Patches #1-#6 are preparations, making things a bit smoother as well. > >> > Patch #7 implements interrupt based completion for admin queue. > >> > >> Hi, Jiri > >> > >> Before this set, I pushed the cvq irq set [1], and the discussion focused on the > >> fact that the newly added irq vector may cause the IO queue to fall back to > >> shared interrupt mode. > >> But it is true that devices implemented according to the specification should > >> not encounter this problem. So what do you think? > > Wait. Please note that admin queue is only created and used by PF virtio > device. And most probably, this is on hypervisor managing the VFs that > are passed to guest VMs. These VFs does not have admin queue. > > Therefore, this is hardly comparable to control vq. Well Parav recently posted patches adding admin queue to VFs, with new "self" group type. > > >> > >> [1] https://lore.kernel.org/all/20240619171708-mutt-send-email-mst@xxxxxxxxxx/ > > > >It's true - this can cause guest to run out of vectors for a variety of > >reasons. > > > >First we have guest irqs - I am guessing avq could use IRQF_SHARED ? > > There is no avq in quest, under normal circumstances. Unless for some > reason somebody passes trough virtio PF into guest. At the moment, but this will change soon. > > >I am not sure why we don't allow IRQF_SHARED for the config > >interrupt though. So I think addressing this part can be deferred. > > > >Second, we might not have enough msix vectors on the device. Here sharing > >with e.g. cvq and further with config interrupt would make sense. > > For cvq irq vector, I believe that sharing with config irq makes sense. > Even for admin queue maybe. But again, admin queue is on PF. I don't > think this is a real concern. > > > > > >Jiri do you think you can help Heng Qi hammer out a solution for cvq? > >I feel this will work will then benefit in a similar way, > >and having us poll aggressively for cvq but not admin commands > >does not make much sense, right? > > > >> > Patch #8 finally removes the admin queue serialization lock. > >> > > >> > Jiri Pirko (8): > >> > virtio_pci: push out single vq find code to vp_find_one_vq_msix() > >> > virtio_pci_modern: treat vp_dev->admin_vq.info.vq pointer as static > >> > virtio: push out code to vp_avq_index() > >> > virtio: create admin queues alongside other virtqueues > >> > virtio_pci_modern: create admin queue of queried size > >> > virtio_pci_modern: pass cmd as an identification token > >> > virtio_pci_modern: use completion instead of busy loop to wait on > >> > admin cmd result > >> > virtio_pci_modern: remove admin queue serialization lock > >> > > >> > drivers/virtio/virtio.c | 28 +---- > >> > drivers/virtio/virtio_pci_common.c | 109 ++++++++++++++------ > >> > drivers/virtio/virtio_pci_common.h | 9 +- > >> > drivers/virtio/virtio_pci_modern.c | 160 ++++++++++++----------------- > >> > include/linux/virtio.h | 2 + > >> > include/linux/virtio_config.h | 2 - > >> > 6 files changed, 150 insertions(+), 160 deletions(-) > >> > > >> > -- > >> > 2.45.1 > >> > > >> > > >
