Hello Don! Am Tue, 13 Dec 2011 01:21:41 +0100 schrieb Andreas Hartmann <andihartmann@xxxxxxxxxxxxxxx>: > Am Mon, 12 Dec 2011 13:36:36 -0500 > schrieb Don Dutile <ddutile@xxxxxxxxxx>: > > > On 12/12/2011 06:15 AM, Andreas Hartmann wrote: > > > Hello! > > > > > > I've got a few questions to a problem, which already was analyzed here > > > sometime ago: > > > http://markmail.org/message/dspovwvzp3wtdrf6#query:+page:1+mid:i2oph4xwfmiknt3y+state:results > > > > > > My situation is a bit different. I do have two PCI cards (a Linksys wlan > > > card and an intel e100 card). Each of these cards should be managed in > > > an own VM. I do have no problems with IRQ sharing (each device does have its own IRQ). > > > > > > I'm using linux 3.0.6, kvm 0.15 and libvirt 0.9.7. Mainboard is: GA-990XA-UD3. > > > > > > > > > The problem is: both cards are behind a PCI-PCI bridge: > > > > > > 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (rev 40) > > > > > > > > > -[0000:00]-+-00.0 ATI Technologies Inc RD890 PCI to PCI bridge (external gfx0 port B) > > > +-00.2 ATI Technologies Inc Device 5a23 > > > +-02.0-[01]--+-00.0 ATI Technologies Inc Device 6759 > > > | \-00.1 ATI Technologies Inc Device aa90 > > > +-04.0-[02]----00.0 Device 1b6f:7023 > > > +-09.0-[03]----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller > > > +-0a.0-[04]----00.0 Device 1b6f:7023 > > > +-11.0 ATI Technologies Inc SB700/SB800 SATA Controller [AHCI mode] > > > +-12.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller > > > +-12.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller > > > +-13.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller > > > +-13.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller > > > +-14.0 ATI Technologies Inc SBx00 SMBus Controller > > > +-14.1 ATI Technologies Inc SB700/SB800 IDE Controller > > > +-14.2 ATI Technologies Inc SBx00 Azalia (Intel HDA) > > > +-14.3 ATI Technologies Inc SB700/SB800 LPC host controller > > > > > > +-14.4-[05]--+-06.0 Intel Corporation 82557/8/9/0/1 Ethernet Pro 100 > > > | \-07.0 RaLink RT2800 802.11n PCI > > > > > You cannot assign two devices behind a (legacy) PCI (not PCIe) bridge to two > > different VMs because PCI devices dont provide a B:D.F in their transaction headers; > > In this case, the PCIe-PCI bridge/switch prepends the bridge's B:D.F in front of > > a transaction that is sourced by 05:06.0 or 05:07.0 . > > > > So, the devices cannot be isolated from each other's (mem) mapping domains in the > > IOMMU, thus, libvirt prevents this (security) violation from being enabled. > > Ok. If I remove the intel card and put in instead a 32 bit PCIe card > like TP-Link TG-3468, I could assign each of these two cards to > different VMs. > > Is this correct? > > [...] > > > > 1. Is there any way to get the devices into different VMs? (I can't put them to another PCI slot as there are just 2 PCI slots on the board.) > > Only if the two slots are behind different PCIe-PCI bridges . > > > > > 2. Is there any fix to prevent the host crash - maybe in a newer version of kvm or kernel? > > don't assign the PCI bridge; libvirt does all the proper bind/unbinding of devices > > for assigned devices, so the above echo steps are unnecessary if you use libvirt. > > If I just remove the intel card, I should be able to assign the wlan > card to one of my VMs. I'll try that. Meanwhile I checked this scenario: I removed the intel card and rebooted. I got the following pci tree: -[0000:00]-+-00.0 ATI Technologies Inc RD890 PCI to PCI bridge (external gfx0 port B) +-00.2 ATI Technologies Inc Device 5a23 +-02.0-[01]--+-00.0 ATI Technologies Inc Device 6759 | \-00.1 ATI Technologies Inc Device aa90 +-04.0-[02]----00.0 Device 1b6f:7023 +-09.0-[03]----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller +-0a.0-[04]----00.0 Device 1b6f:7023 +-11.0 ATI Technologies Inc SB700/SB800 SATA Controller [AHCI mode] +-12.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller +-12.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller +-13.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller +-13.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller +-14.0 ATI Technologies Inc SBx00 SMBus Controller +-14.1 ATI Technologies Inc SB700/SB800 IDE Controller +-14.2 ATI Technologies Inc SBx00 Azalia (Intel HDA) +-14.3 ATI Technologies Inc SB700/SB800 LPC host controller +-14.4-[05]----07.0 RaLink RT2800 802.11n PCI +-14.5 ATI Technologies Inc SB700/SB800 USB OHCI2 Controller +-15.0-[06]-- +-16.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller +-16.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller +-18.0 Advanced Micro Devices [AMD] Device 1600 +-18.1 Advanced Micro Devices [AMD] Device 1601 +-18.2 Advanced Micro Devices [AMD] Device 1602 +-18.3 Advanced Micro Devices [AMD] Device 1603 +-18.4 Advanced Micro Devices [AMD] Device 1604 \-18.5 Advanced Micro Devices [AMD] Device 1605 But this doesn't work, too: I had to manually unbind the bridge (as described in the mail before - it was not necessary to manually unbind the device itself). On closing the VM, the complete host crashed with this screen (excerpt): amd_iommu_domain_destroy iommu_domain_free kvm_iommu_unmap_guest kvm_arch_destroy_vm kvm_destroy_vm kvm_vcpu_release __fput flip_close put_files_struct do_exit do_group_exit get_signal_to_deliver do_signal do_notify_resume int_signal Seams to even not work if there's just one (legal) PCI device plugged in. Could somebody please shed some light on this problem? Thank you, kind regards, Andreas Hartmann -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
