On Tue, 2022-09-13 at 15:09 +0100, Jiaxun Yang wrote: > > So I've got a couple of questions about moving forward: > > 1. Is this hardware behaviour a violation of PCIe spec? > > 2. Is it expected for IOMMU to handle those ill-formed MSI requests? If not is > > it possible to get this workaround upstreamed? > > 3. For Qualcomm guys, is it possible to fix this behaviour in ath11k driver or firmware? The IOMMU can't handle those requests. The "compatibility format" MSI directly identifies the *guest* APICID and vector to deliver. In an ideal world maybe the IOMMU could have been designed to handle those MSIs directly. The PCIe requester-id could identify the guest and thus route the interrupt via appropriately-designed tables. Then the guest could be permitted to compose the intended MSI message for itself and write it directly to the device's MSI(X) table. But that isn't how it was done. Instead, the device has to be programmed to submit "remappable format" MSIs instead. That's what the IOMMU needs to look up the translation (and the contents of the remappable format MSI are mostly just an index into the IOMMU table). So instead of letting the guest program the MSI directly into the hardware, a VMM will *trap* the writes to the MSI(X) table and create an interrupt route, then write the *actual* hardware with a remappable format MSI that will trigger the correct guest interrupt. Your best bet is probably to have the hypervisor trap accesses to the ath11k_ce_srng_msi_ring_params_setup and ath11k_dp_srng_msi_setup registers and do the same trick of allocating a host IRQ route and writing the converted MSI message. The alternative is to build some kind of enlightenment to let the guest ask the hypervisor to tell it the correct remappable format message to use... but that's going to get very horrid very quickly. If your hardware allows you to read the MSI(X) table via some other indirect route which isn't trapped by the hypervisor to give you back the "original" thing you wrote, perhaps you could write to an unused MSI and then read back what the hardware *actually* contains... then write that to the special register? But that's kind of awful.
Attachment:
smime.p7s
Description: S/MIME cryptographic signature
