On Tue, 2012-06-05 at 12:39 +0200, Andreas Hartmann wrote: > Hello Alex, > > thanks for your efforts! > > Maybe, you already know that I'm suffering the same problem :-( with a > GA-990XA-UD3 (990X chipset). My system is a GA-990FXA-UD3, so just a slightly different version of the chipset. > On Mon, 04 Jun 2012 21:44:05 -0600 > Alex Williamson <alex.williamson@xxxxxxxxxx> wrote: > [...] > > I have a setup with an AMD 990FX system and a spare PVR-350 card that I > > installed to reproduce. The sad answer is that it's nearly impossible > > to assign PCI devices on these systems due to the aliasing of devices > > below the PCIe-to-PCI bridge (PCIe devices are much, much easier to > > assign). If you boot with amd_iommu_dump, you'll see some output like > > this: > > > > AMD-Vi: DEV_ALIAS_RANGE devid: 05:00.0 flags: 00 devid_to: 00:14.4 > > AMD-Vi: DEV_RANGE_END devid: 05:1f.7 > > here: > AMD-Vi: DEV_ALIAS_RANGE devid: 06:00.0 flags: 00 devid_to: 00:14.4 > AMD-Vi: DEV_RANGE_END devid: 06:1f.7 > > > This means according to your description, the following devices have to > be unbound here, too (0000:06:07.0 is the device I want to passthrough): > > 0000:00:14.0 -> ../../../../devices/pci0000:00/0000:00:14.0 > 0000:00:14.1 -> ../../../../devices/pci0000:00/0000:00:14.1 > 0000:00:14.2 -> ../../../../devices/pci0000:00/0000:00:14.2 > 0000:00:14.3 -> ../../../../devices/pci0000:00/0000:00:14.3 > 0000:00:14.4 -> ../../../../devices/pci0000:00/0000:00:14.4 > 0000:00:14.5 -> ../../../../devices/pci0000:00/0000:00:14.5 > 0000:06:07.0 -> ../../../../devices/pci0000:00/0000:00:14.4/0000:06:07.0 Yep. > These are the following additional devices: > > 00:14.0 SMBus: Advanced Micro Devices [AMD] nee ATI SBx00 SMBus Controller (rev 42) > 00:14.1 IDE interface: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 IDE Controller (rev 40) (prog-if 8a [Master SecP PriP]) > 00:14.2 Audio device: Advanced Micro Devices [AMD] nee ATI SBx00 Azalia (Intel HDA) (rev 40) > 00:14.3 ISA bridge: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 LPC host controller (rev 40) > 00:14.4 PCI bridge: Advanced Micro Devices [AMD] nee ATI SBx00 PCI to PCI Bridge (rev 40) (prog-if 01 [Subtractive decode]) > 00:14.5 USB controller: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI2 Controller (prog-if 10 [OHCI]) My system shows this for lspci -n -s 14. 00:14.0 0c05: 1002:4385 (rev 42) 00:14.2 0403: 1002:4383 (rev 40) 00:14.3 0601: 1002:439d (rev 40) 00:14.4 0604: 1002:4384 (rev 40) 00:14.5 0c03: 1002:4399 Can you confirm yours matches and fill in 14.1 so we have the vendor:device IDs for these. > Among them is the sound device and the USB device - no good idea to > disable them on a desktop. You can always blacklist drivers if the device is unique or unbind devices in an rc.local type script. I'm hoping that Joerg or someone else from AMD can tell us this devices does not allow internal peer-to-peer between functions. Then we can add it to the device specific ACS checks and you wouldn't need to worry about displacing all the other 14.x functions. > Disabling 00:14.1 seems to be possible (rmmod pata_atiixp works), but > I'm getting errors after rebooting the system (the filesystems haven't > been cleanly unmounted during shutdown). Did you have a disk off that controller? My system doesn't expose function 1, but is the same otherwise. > Anyway, I would have tested it, but I'm getting a compile error while > compiling qemu. It complains about missing pci/header.h while > processing hw/vfio_pci.c:49:24. I can't find any pci/header.h. Where > should it come from? It is from pciutils-devel for me. I'll see what I'm getting out of there and try to remove it. > > The downside is that VFIO is strict about > > multifunction devices supporting ACS to prevent peer-to-peer between > > domains, so will require all of the 14.x devices to be bound to pci-stub > > as well. On my system, this includes an smbus controller, audio device, > > lpc controller, and usb device. If AMD could confirm this device > > doesn't allow peer-to-peer between functions, we could relax this > > requirement a bit. > > Please Joerg, could you take a look at this problem? Joerg, the question is whether the multifunction device above allows peer-to-peer between functions that could bypass the iommu. If not, we can make it the first entry in device specific acs enabled function I proposed: https://lkml.org/lkml/2012/5/30/438 and it would greatly simplify assigning PCI devices on these systems with VFIO. Thanks, Alex -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
