Hello kvm-list! I sent the following text to Don with some additional detailed log files. If somebody else need them too, I can provide them with pn. Kind regards, Andreas Hartmann Andreas Hartmann schrieb: > Hello Don, > > thank you for your reply! > > I just want to describe in short the two problems I encounter (my > description was a bit chaotic :-) I hope it's better now :-)). I > attached details in two files (dmesg.bz2 and lspci.bz2. It's the > untouched raw output - no unbind or anything else had been done). > > General hardware setup for a quick overview: > > -[0000:00]-+-00.0 ATI Technologies Inc RD890 PCI to PCI bridge (external gfx0 port B) > +-00.2 ATI Technologies Inc Device 5a23 > +-02.0-[01]--+-00.0 ATI Technologies Inc Device 6759 > | \-00.1 ATI Technologies Inc Device aa90 > +-04.0-[02]----00.0 Device 1b6f:7023 > +-05.0-[03]----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller > +-09.0-[04]----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller > +-0a.0-[05]----00.0 Device 1b6f:7023 > +-11.0 ATI Technologies Inc SB700/SB800 SATA Controller [AHCI mode] > +-12.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller > +-12.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller > +-13.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller > +-13.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller > +-14.0 ATI Technologies Inc SBx00 SMBus Controller > +-14.1 ATI Technologies Inc SB700/SB800 IDE Controller > +-14.2 ATI Technologies Inc SBx00 Azalia (Intel HDA) > +-14.3 ATI Technologies Inc SB700/SB800 LPC host controller > +-14.4-[06]----07.0 RaLink RT2800 802.11n PCI > +-14.5 ATI Technologies Inc SB700/SB800 USB OHCI2 Controller > +-15.0-[07]-- > +-16.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller > +-16.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller > +-18.0 Advanced Micro Devices [AMD] Device 1600 > +-18.1 Advanced Micro Devices [AMD] Device 1601 > +-18.2 Advanced Micro Devices [AMD] Device 1602 > +-18.3 Advanced Micro Devices [AMD] Device 1603 > +-18.4 Advanced Micro Devices [AMD] Device 1604 > \-18.5 Advanced Micro Devices [AMD] Device 1605 > > The relevant devices are (for problem 1): > > 06:07.0 Network controller: RaLink RT2800 802.11n PCI > Subsystem: Linksys Device 0067 > Flags: bus master, slow devsel, latency 32, IRQ 21 > Memory at fd8e0000 (32-bit, non-prefetchable) [size=64K] > Capabilities: [40] Power Management version 3 > Kernel driver in use: rt2800pci > > with the PCI-PCI bridge above: > > 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (rev 40) (prog-if 01 [Subtractive decode]) > Flags: bus master, VGA palette snoop, 66MHz, medium devsel, latency 64 > Bus: primary=00, secondary=06, subordinate=06, sec-latency=64 > I/O behind bridge: 00009000-00009fff > Memory behind bridge: fd800000-fd8fffff > Prefetchable memory behind bridge: fd700000-fd7fffff > > > > and (for problem 2): > > 03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 01) > Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller > Flags: bus master, fast devsel, latency 0, IRQ 10 > I/O ports at de00 [size=256] > Memory at fdbff000 (64-bit, non-prefetchable) [size=4K] > [virtual] Expansion ROM at fda00000 [disabled] [size=128K] > Capabilities: [40] Power Management version 2 > Capabilities: [48] Vital Product Data > Capabilities: [50] MSI: Enable- Count=1/2 Maskable- 64bit+ > Capabilities: [60] Express Endpoint, MSI 00 > Capabilities: [84] Vendor Specific Information: Len=4c <?> > Capabilities: [100] Advanced Error Reporting > Capabilities: [12c] Virtual Channel > Capabilities: [148] Device Serial Number f1-11-00-00-68-4c-e0-00 > Capabilities: [154] Power Budgeting <?> > > 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06) > Subsystem: Giga-byte Technology GA-EP45-DS5 Motherboard > Flags: bus master, fast devsel, latency 0, IRQ 10 > I/O ports at ce00 [size=256] > Memory at fd6ff000 (64-bit, prefetchable) [size=4K] > Memory at fd6f8000 (64-bit, prefetchable) [size=16K] > Capabilities: [40] Power Management version 3 > Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ > Capabilities: [70] Express Endpoint, MSI 01 > Capabilities: [b0] MSI-X: Enable- Count=4 Masked- > Capabilities: [d0] Vital Product Data > Capabilities: [100] Advanced Error Reporting > Capabilities: [140] Virtual Channel > Capabilities: [160] Device Serial Number 12-34-56-78-12-34-56-78 > > > ============================================================================================================================ > > > 1. problem: > > Wlan RaLink RT2800 802.11n is a legacy PCI device behind > a PCI-PCI bridge and should be assigned to VM_a, but passthrough doesn't > work because libvirt claims, that the device or resource would be busy > (but the device is unbound correctly). > > There is no other device in the tree under the bridge (for details > please see the attached log file). > > This is the entry in the xml-file I use: > > <hostdev mode='subsystem' type='pci' managed='yes'> > <source> > <address domain='0x0000' bus='0x06' slot='0x07' function='0x0'/> > </source> > </hostdev> > > The VM is started with "virsh start VM_a". > > I was able to start VM_a if the PCI-PCI bridge was unbound manually > before VM_a was started (yes, I really know, that libvirt does the > unbind job for me - but it doesn't work as expected :-). > The bridge itself wasn't assigned to VM_a): > > echo "1002 4384" > /sys/bus/pci/drivers/pci-stub/new_id > echo "0000:00:14.4" > /sys/bus/pci/devices/0000:00:14.4/driver/unbind > > Unfortunately, the host is locked up on shutdown of VM_a. > > That's the trace of the locked host: > > amd_iommu_domain_destroy > iommu_domain_free > kvm_iommu_unmap_guest > kvm_arch_destroy_vm > kvm_destroy_vm > kvm_vcpu_release > __fput > flip_close > put_files_struct > do_exit > do_group_exit > get_signal_to_deliver > do_signal > do_notify_resume > int_signal > > > But I think this should be a valid setup according [1]: > > "Notes: > VT-d spec specifies that all conventional PCI devices behind a PCIe-to > PCI/PCI-X bridge or conventional PCI bridge can only be collectively > assigned to the same guest. PCIe devices do not have this restriction." > > [1] http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM > > > > > 2. problem: > > Next I tried to assign the PCIe ethernet card, TP-Link TG-3468 > (device 03:00.0), to VM_b. > > This card does have exactly the same PCIID as the onboard network > interface. If I put it (or even both ethernet network devices) to VM_b, > the machine is locked up completely directly on start up of the "virsh > start VM_b" command. > > Problem seems to be. that the kernel (-> for pci passthrough) can't > distinguish those two devices because of their same PCIID (there were > always both devices unbound, even if there should be just one device > unound). > > The xml-entry is: > > <hostdev mode='subsystem' type='pci' managed='yes'> > <source> > <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> > </source> > </hostdev> > > If I remove the additional PCIe TP-Link card, I'm able to passthrough > the onboard ethernet card to VM_b without any problem. > > > > > I hope I could clarify now the two problems I encounter. > > > Thank you very much for your time! > Andreas -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
