Dear all, sorry for bringing this up again, but after having put 60 hours in trying to find a solution on my own, I am stuck. A while ago, I had an interesting discussion on that list where I was shown how to make a Marvell 9128 work with the IOMMU, which finally enabled me to pass through that device into a Windows guest via vfio_pci. A part of the discussion and the associated patch are here: https://patchwork.kernel.org/patch/10178867/ Basically, I have passed through that Marvell 9128 to a Windows guest by doing the following steps: - Configure the module system so that vfio_pci gets loaded before the ahci drivers; - Configure the module system so that vfio_pci is loaded with the right options (nointxmask=1 ids=...) This makes sure that the Marvell 9128 is grabbed by vfio_pci automatically during the boot process. It works as intended; lspci shows that the Marvell 9128 is indeed bound to vfio_pci. - Check that the Marvell 9128 is the only device in its IOMMU group (actually, there is an additional PCIe bridge, but that does not matter according to vfio's documentation). - Since vfio_pci has been assigned an interrupt which already had been bound to another device, disable the other device (i.e. unbind it from its driver on the host). - In the command line which starts the VM, add -device vfio-pci,host=02:00.0,id=marvell9128,bus=pci.0,addr=03.0 This has worked correctly for four months or so, with the Marvell 9128 showing good transfer rates and running absolutely stable in the Windows guest (basically full speed of the HDDs attached, 350 GB/day backup data without a single error (backup files are checksummed)). This was with a kernel which debian called 4.9.0-4... or 4.9.65, respectively. Recently, I decided to take debian's offer and upgrade the kernel (mainly due to Meltdown and Spectre) to what debian called 4.9.0-6... or 4.9.82, respectively. >From then on, I still could pass the Marvell 9128 to the Windows guest, but its transfer rates in the guest became ridiculous low (about 2 MB/s). Furthermore, when starting the VM, the Seabios hung about a minute or so before starting the boot process. This is a phenomenon I had already encountered with the previous kernel, but only until I had disabled the device which had its interrupt shared with the vfio_pci module, and until I had given nointxmask=1 to the vfio_pci module. Unfortunately, this remedy is not effective with the new kernel. Whatever I do, the Seabios first hangs when starting the VM, then the VM boots up, then the Marvell 9128 is unusable in the VM due to its low transfer rates. I have double checked multiple times that the (module and other) configuration is exactly as before. I even have compiled the newest stable vanilla kernel 4.16.5, but using this one didn't change anything (by the way, this test took a lot of time because that server's root fs is zfs, but the zfs infrastructure which ships with debian stretch is too old for that kernel, so I had to compile this myself as well and inject it into the system). I originally thought that the misbehavior might have to do with the anti-meltdown and anti-spectre measures in the new debian kernel. So I compiled the new debian kernel as well as the vanilla kernel with RETPOLINE disabled and PAGE_TABLE_ISOLATION disabled. Additionally, as I couldn't find where GENERIC_CPU_VULNERABILITIES can be configured in make menuconfig, I changed drivers/base/cpu.c so that it didn't have any effect. This did not change anything either - with both kernels, Seabios still hung when starting the VM, and the transfer rates of the Marvell 9128 in the guest were still unusable. Now I am completely stuck. Can anybody imagine what have might been changed in the kernel (shortly after 4.9.65) which could cause this behavior? Thank you very much again, Binarus
