On 07.01.2018 07:22, Alex Williamson wrote: > For the system you're using there is no integrated graphics on the > processor, so "igfx_off" does nothing. However, using that as the only > parameter to intel_iommu= is basically the same as not providing an > intel_iommu= option at all, ie. the IOMMU is not enabled, thus... At first, thank you very much for spending time on that problem! I see. This is the opposite logic than what I would have expected :-), and it explains the behavior (I would have expected that the option would be turned on if encountered in any case, and that inappropriate option parameters would be ignored instead of turning the option off). Additionally, I have misunderstood the following passage "If you encounter issues with graphics devices, you can try adding option intel_iommu=igfx_off to turn off the integrated graphics engine. If this fixes anything, please ensure you file a bug reporting the problem." from the following document: https://www.kernel.org/doc/Documentation/Intel-IOMMU.txt The wording "If this fixes anything" made me somehow think that igfx_off in rare cases might do something mysterious even to devices which are not integrated graphics devices. Thank you very much for clearing that up and for explaining how intel_iommu works. >> This message is to report that I can't pass through the SATA controller in question to a VM when the kernel has been booted with intel_iommu=igfx_off. Things are going wrong already during the boot process. dmesg | grep vfio says: >> >> [ 1.074239] vfio-pci: probe of 0000:02:00.0 failed with error -22 >> [ 1.074309] vfio_pci: add [1b4b:9128[ffff:ffff]] class 0x000000/00000000 > > vfio-pci won't probe the device because it doesn't have an IOMMU. This is Murphy's law. I have thought a long time about buying another controller with another chipset and obviously made the wrong choice. I should have read the controller's datasheet before buying, but understanding it would surely have cost me a huge amount of time. The controller was about 30 EUR (about 35 US$), so spending two days on understanding that datasheet wouldn't have made a lot of sense ... > Marvell SATA controllers are notoriously bad with IOMMUs, they perform > DMA using a requester ID other than the actual PCI address of the > device. For instance, the SATA controller is often at PCI function > address 0, but function address 1 is used for the requester ID. The > IOMMU uses the requester ID to index the DMA translations for the > device, so getting this fundamental aspect of the transaction wrong in > hardware is often fatal for doing any DMA at all when the IOMMU is > enabled. This should cause all sorts of "DMAR" faults during boot when > VT-d is enabled. Thanks for the in-depth explanation of the problem. I have never read the Marvell datasheets nor the IOMMU specs until now. I wish I had some more time; that whole thing is very interesting. > We do have quirks to handle various Marvell chips, but not this one. > If you're interested in building a kernel, I can provide a patch to > test, though I suspect the reason we don't already have a patch for > this device might be conflicting results. Your other option is to pick > a SATA controller that isn't so broken. Thanks, Initially, I didn't want to build a new kernel on that machine (I don't have a second one at hands with identical hardware). But I am quite impressed by the KVM project in general, and that you already can offer a patch for testing in particular, so I'll do it. I would be glad if I could be of any help to the KVM project. Could you please provide me instructions? Can your patch be applied to 4.8 or 4.9, or only to the newest kernel? One additional question (to get my problem solved in any case): Could you recommend a SATA controller (chip) which works flawlessly in that respect? I just need at least one SATA channel (but better two) passed through into that VM - no raid, mirroring, striping or other fancy things needed. Thank you very much, Binarus
