On Thu, 24 Nov 2022 18:00:44 +0000, Jason Gunthorpe <jgg@xxxxxxxx> wrote: > > On Wed, Nov 23, 2022 at 09:42:36AM +0800, chenxiang via wrote: > > From: Xiang Chen <chenxiang66@xxxxxxxxxxxxx> > > > > Currently the number of MSI vectors comes from register PCI_MSI_FLAGS > > which should be power-of-2 in qemu, in some scenaries it is not the same as > > the number that driver requires in guest, for example, a PCI driver wants > > to allocate 6 MSI vecotrs in guest, but as the limitation, it will allocate > > 8 MSI vectors. So it requires 8 MSI vectors in qemu while the driver in > > guest only wants to allocate 6 MSI vectors. > > > > When GICv4.1 is enabled, it iterates over all possible MSIs and enable the > > forwarding while the guest has only created some of mappings in the virtual > > ITS, so some calls fail. The exception print is as following: > > vfio-pci 0000:3a:00.1: irq bypass producer (token 000000008f08224d) registration > > fails:66311 > > With Thomas's series to make MSI more dynamic this could spell future > problems, as future kernels might have different ordering. Enabling MSIs on the endpoint before they are programmed in the interrupt controller? I don't think that's a realistic outcome. > It is just architecturally wrong to tie the MSI programming at the PCI > level with the current state of the guest's virtual interrupt > controller. There is no architectural ties between the two at all. There is an optimisation that allows direct injection if you do it in a non braindead order. Nothing breaks if you don't, you just have wasted memory, performance, power and area. You're welcome. > Physical hardware doesn't do this, virtual emulation shouldn't either. If you want to fix VFIO, be my guest. My rambling about the sorry state of this has been in the kernel for 5 years (ed8703a506a8). > People are taking too many liberties with trapping the PCI MSI > registers through VFIO. :( Do you really want to leave access to the MSI BAR to userspace? The number of ways this can go wrong is mind-boggling. Starting with having to rebuild the interrupt translation tables on the host side to follow what the guest does, instead of keeping the two independent. M. -- Without deviation from the norm, progress is not possible.
