On Thu, Nov 17, 2022 at 1:48 PM Yongji Xie <xieyongji@xxxxxxxxxxxxx> wrote:
>
> On Thu, Nov 17, 2022 at 11:37 AM Jason Wang <jasowang@xxxxxxxxxx> wrote:
> >
> > On Wed, Nov 16, 2022 at 3:46 PM Yongji Xie <xieyongji@xxxxxxxxxxxxx> wrote:
> > >
> > > On Wed, Nov 16, 2022 at 3:11 PM Jason Wang <jasowang@xxxxxxxxxx> wrote:
> > > >
> > > > On Tue, Nov 15, 2022 at 10:49 AM Yongji Xie <xieyongji@xxxxxxxxxxxxx> wrote:
> > > > >
> > > > > On Mon, Nov 14, 2022 at 4:55 PM Jason Wang <jasowang@xxxxxxxxxx> wrote:
> > > > > >
> > > > > > On Mon, Nov 14, 2022 at 4:20 PM Yongji Xie <xieyongji@xxxxxxxxxxxxx> wrote:
> > > > > > >
> > > > > > > On Mon, Nov 14, 2022 at 3:58 PM Jason Wang <jasowang@xxxxxxxxxx> wrote:
> > > > > > > >
> > > > > > > > On Mon, Nov 14, 2022 at 3:16 PM Xie Yongji <xieyongji@xxxxxxxxxxxxx> wrote:
> > > > > > > > >
> > > > > > > > > Add sysfs interface for each vduse virtqueue to setup
> > > > > > > > > irq affinity. This would be useful for performance
> > > > > > > > > tuning, e.g., mitigate the virtqueue lock contention
> > > > > > > > > in virtio block driver.
> > > > > > > >
> > > > > > > > Do we have any perforamnce numbers for this?
> > > > > > > >
> > > > > > >
> > > > > > > Almost 50% improvement (600k iops -> 900k iops) in the high iops
> > > > > > > workloads. I have mentioned it in the cover-letter.
> > > > > >
> > > > > > For some reason, I miss that.
> > > > > >
> > > > > > I also wonder if we can do this automatically, then there's no need to
> > > > > > play with sysfs which is kind of a burden for the management layer.
> > > > > >
> > > > >
> > > > > This is hard to do since vduse doesn't know which cpu should be bound
> > > > > for a certain virtqueue.
> > > >
> > > > Probably via the kick_vq()? It probably won't work when notification
> > > > is disabled. But we need to think a little bit more about this.
> > >
> > > Yes, another problem is that this way can only work when the cpu and
> > > virtqueue are 1:1 mapping. It's still hard to decide which cpu to bind
> > > in the N:1 mapping case.
> >
> > This is the same situation as what you propose here. I think it would
> > be better to use cpumask instead of cpu id here.
> >
>
> If so, we need to know which cpu to bind for one virtqueue. Do you
> mean using the cpu who kicks the virtqueue?
I meant you're using:
int irq_affinity;
This seems to assume that the callback can only be delivered to a
specific cpu. It would make more sense to use cpumask_t. This may have
broader use cases.
>
> > >
> > > So I think it could be an optimization, but the sysfs interface is still needed.
> > >
> > > > Requiring management software to do ad-hoc running just for VDUSE
> > > > seems not easy.
> > > >
> > >
> > > I'm not sure. In the kubernetes environment, something like a CSI/CNI
> > > plugin can do it.
> >
> > Only works when the process is bound to a specific cpu. If a process
> > is migrated to another CPU, it would be hard to track.
> >
>
> OK, I see. Seems like there's no good way to handle this case.
Yes, using cpumask_t might improve things a little bit.
> Maybe
> it's better to leave it as it is.
It would be better to think of an automatic method to do this as
affinity managed irq used by virtio-pci (not sure how hard it is
though).
Thanks
>
> Thanks,
> Yongji
>
_______________________________________________
Virtualization mailing list
Virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
