On Sat, 6 Jan 2018 12:55:36 +0100 Binarus <lists@xxxxxxxxxx> wrote: > Dear all, > > in my first two messages from today, I have described how intel_iommu=on makes the AHCI / SATA driver misbehave when trying to initialize an additional PCIe SATA controller card on one of our servers, and how intel_iommu=igfx_off changed that situation. For the system you're using there is no integrated graphics on the processor, so "igfx_off" does nothing. However, using that as the only parameter to intel_iommu= is basically the same as not providing an intel_iommu= option at all, ie. the IOMMU is not enabled, thus... > This message is to report that I can't pass through the SATA controller in question to a VM when the kernel has been booted with intel_iommu=igfx_off. Things are going wrong already during the boot process. dmesg | grep vfio says: > > [ 1.074239] vfio-pci: probe of 0000:02:00.0 failed with error -22 > [ 1.074309] vfio_pci: add [1b4b:9128[ffff:ffff]] class 0x000000/00000000 vfio-pci won't probe the device because it doesn't have an IOMMU. > 1b4b:9128 is indeed the correct Vendor / Device ID, and it finds the device correctly (bus 2, device 0, function 0 is exactly the place where the controller resides, according to lspci after having booted without any kernel parameters). But although it finds the device, it fails when probing it (I admit that I have no clue what this means). Marvell SATA controllers are notoriously bad with IOMMUs, they perform DMA using a requester ID other than the actual PCI address of the device. For instance, the SATA controller is often at PCI function address 0, but function address 1 is used for the requester ID. The IOMMU uses the requester ID to index the DMA translations for the device, so getting this fundamental aspect of the transaction wrong in hardware is often fatal for doing any DMA at all when the IOMMU is enabled. This should cause all sorts of "DMAR" faults during boot when VT-d is enabled. We do have quirks to handle various Marvell chips, but not this one. If you're interested in building a kernel, I can provide a patch to test, though I suspect the reason we don't already have a patch for this device might be conflicting results. Your other option is to pick a SATA controller that isn't so broken. Thanks, Alex
