Hi! Some time ago already, we got graphics card pci passthrough working on a friend's computer, an ASRock Z68 Extreme4 Gen3 based PC with an Intel i7-2600, running on a Linux 3.2 kernel, with two Gigabyte GV-R677SL-1GD (AMD Radeon HD 6770) graphics cards in CrossFireX mode. To get things to work we had to: - Boot with intel_iommu=yes. This broke the onboard Marvell 88SE9120 controller (an AHCI controller) with IOMMU related kernel errors, and we had to move some drives around. - Blacklist the radeon audio driver so that the graphics cards aren't in use on the host. - Apply the patches at the end of this mail to kvm and SeaBIOS to allow for more BAR space under 4G. (The relevant BARs on the graphics cards _are_ 64 bit BARs, but kvm seemed to turn those into 32 bit BARs in the guest.) - Apply the hacky patch at the end of this mail to SeaBIOS to always skip initialising the Radeon's option ROMs, or the VM would hang inside the Radeon option ROM if you boot the VM without the default cirrus video. With these patches, the Windows 7 guest seemed to work OK on basic testing, allowing playing of all 3d games we tried, including in CrossFireX mode. We ran into the following issues: - The AMD Catalyst graphics card drivers bluescreen the VM on win7 boot if you boot with a CPU family/model/stepping that it does not like and one or more Radeon cards passed through to the VM. (Setting the guest family/model/stepping to be equal to the host's CPU's fixes the bluescreen and allows booting to complete.) - AMD Catalyst 12.4 worked okay, but the 12.6 beta would bluescreen the VM. - Playing video in Flash (e.g. youtube) and XBMC gives a green rectangle where the video is supposed to be. This was fixable in both cases by turning off hardware acceleration (in Flash) or "dxva2" (in xbmc). A problem with these symptoms is noted in the AMD Catalyst driver release notes, but the same problem does not seem to appear when running Windows 7 directly on the host. (Could this be related to skipping the option ROM?) We also tried passing through an NVidia graphics card (GTX 260) instead of the Radeons, but that would crash the VM the couple of times we tried, and we didn't look into this further. The setup (display on the right assigned to the Linux host, display on the left assigned to the Windows guest): http://i.imgur.com/idtVH.jpg 3DMark running on Windows host: http://i.imgur.com/nvbQr.jpg 3DMark running on Windows as a VM: http://i.imgur.com/vuL7H.jpg (Running with only one graphics card gives about half the 3dmark score, so CrossFireX does appear to be working in both cases.) cheers, Lennert KVM patch for more <4G PCI BAR space: diff -ur qemu-kvm-1.0+noroms/hw/pc_piix.c ../qemu-kvm-1.0+noroms/hw/pc_piix.c --- qemu-kvm-1.0+noroms/hw/pc_piix.c 2011-12-04 11:38:06.000000000 +0100 +++ ../qemu-kvm-1.0+noroms/hw/pc_piix.c 2012-06-09 16:24:14.186327304 +0200 @@ -112,9 +112,9 @@ kvmclock_create(); } - if (ram_size >= 0xe0000000 ) { - above_4g_mem_size = ram_size - 0xe0000000; - below_4g_mem_size = 0xe0000000; + if (ram_size >= 0xc0000000 ) { + above_4g_mem_size = ram_size - 0xc0000000; + below_4g_mem_size = 0xc0000000; } else { above_4g_mem_size = 0; below_4g_mem_size = ram_size; Corresponding SeaBIOS patch: diff --git a/src/acpi-dsdt.dsl b/src/acpi-dsdt.dsl index 15299ee..31e7102 100644 --- a/src/acpi-dsdt.dsl +++ b/src/acpi-dsdt.dsl @@ -170,10 +170,10 @@ DefinitionBlock ( ,, , AddressRangeMemory, TypeStatic) DWordMemory (ResourceProducer, PosDecode, MinFixed, MaxFixed, NonCacheable, ReadWrite, 0x00000000, // Address Space Granularity - 0xE0000000, // Address Range Minimum + 0xC0000000, // Address Range Minimum 0xFEBFFFFF, // Address Range Maximum 0x00000000, // Address Translation Offset - 0x1EC00000, // Address Length + 0x3EC00000, // Address Length ,, PW32, AddressRangeMemory, TypeStatic) }) Name (CR64, ResourceTemplate () diff --git a/src/config.h b/src/config.h index 3a70867..b305d9a 100644 --- a/src/config.h +++ b/src/config.h @@ -44,9 +44,9 @@ #define BUILD_SMM_INIT_ADDR 0x38000 #define BUILD_SMM_ADDR 0xa8000 #define BUILD_SMM_SIZE 0x8000 -#define BUILD_MAX_HIGHMEM 0xe0000000 +#define BUILD_MAX_HIGHMEM 0xc0000000 -#define BUILD_PCIMEM_START 0xe0000000 +#define BUILD_PCIMEM_START 0xc0000000 #define BUILD_PCIMEM_END 0xfec00000 /* IOAPIC is mapped at */ #define BUILD_PCIMEM64_START 0x8000000000ULL #define BUILD_PCIMEM64_END 0x10000000000ULL SeaBIOS patch to always skip Radeon option ROM initialization: diff --git a/src/optionroms.c b/src/optionroms.c index 00697b2..857956a 100644 --- a/src/optionroms.c +++ b/src/optionroms.c @@ -440,6 +440,8 @@ vga_setup(void) foreachpci(pci) { if (!is_pci_vga(pci)) continue; + if (pci->vendor == 0x1002 && pci->device == 0x68ba) + continue; vgahook_setup(pci); init_pcirom(pci, 1, NULL); break; -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
