Hi Alex,
Thanks for your reply. The pci devices in question are proprietary FPGAs. Here is the lspci -tv output:
-bash-4.1# lspci -tv
-[0000:00]-+-00.0 Intel Corporation Sandy Bridge DRAM Controller
+-01.0-[01-04]----00.0-[02-04]--+-01.0-[03]----00.0 Broadcom Corporation Device b850
| \-02.0-[04]----00.0 Broadcom Corporation Device b850
+-01.1-[05]--
+-06.0-[06]--+-00.0 Intel Corporation Device 0434
| +-00.1 Intel Corporation Device 0438
| +-00.2 Intel Corporation Device 0438
| +-00.3 Intel Corporation Device 0436
| \-00.4 Intel Corporation Device 0436
+-1d.0 Intel Corporation Device 2334
+-1f.0 Intel Corporation Device 2310
+-1f.2 Intel Corporation Device 2323
+-1f.3 Intel Corporation Device 2330
+-1f.4 Intel Corporation Device 2331
+-1f.6 Intel Corporation Device 2332
\-1f.7 Intel Corporation Device 2360
My qemu command line is as follows:
qemu-system-x86_64 -M q35 --enable-kvm -m 2048 -nographic -vga std -usb -drive file=<IMG file>,if=none,id=drive-sata-disk0,format=raw -device ahci,id=ahci -device ide-drive,bus=ahci.0,drive=drive-sata-disk0,id=sata-disk0,bootindex=1 -device pci-assign,host=04:00.0 -device pci-assign,host=03:00.0
The PCIe bridge is a PLX 8613 device:
01:00.0 PCI bridge: PLX Technology, Inc. PEX 8613 12-lane, 3-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
02:01.0 PCI bridge: PLX Technology, Inc. PEX 8613 12-lane, 3-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
02:02.0 PCI bridge: PLX Technology, Inc. PEX 8613 12-lane, 3-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
As shown by the lspci -tv output, each of the PCI device being passed through is connected to one of the downstream ports of the PLX PCI bridge.
Thanks,
Ganesh
On Mar 19, 2013, at 3:28 PM, Alex Williamson wrote:
> On Tue, 2013-03-19 at 13:30 -0700, Ganesh Narayanaswamy wrote:
>> Hi,
>>
>> I am running qemu with kvm and VT-d enabled and a couple of PCI
>> devices assigned to the guest VM. Both host and guest are running
>> linux 2.6 kernel.
>>
>> The passthrough works fine, but when I exit the VM, the host kernel
>> crashes with the following backtrace:
>>
>> <4>[ 5569.836893] Process qemu-system-x86 (pid: 2925, threadinfo ffff8801f5f40000, task ffff88024fa28720)
>> <0>[ 5569.944946] Stack:
>> <4>[ 5569.968845] ffff8801f5f41aa8 ffffffff811a45fb ffff88024f04b680 ffff88024f049980
>> <4>[ 5570.057156] ffff88024f04b680 ffff88024f049988 ffff8801f5f41b08 ffffffff811a6371
>> <4>[ 5570.145470] ffff8801f5f41ad8 ffffffff81391045 0000000000000246 ffff88024f049990
>> <0>[ 5570.233785] Call Trace:
>> <4>[ 5570.262880] [<ffffffff811a45fb>] iommu_detach_dependent_devices+0x25/0x91
>> <4>[ 5570.344958] [<ffffffff811a6371>] vm_domain_exit+0xf8/0x28b
>> <4>[ 5570.411457] [<ffffffff81391045>] ? sub_preempt_count+0x92/0xa6
>> <4>[ 5570.482106] [<ffffffff811a651a>] intel_iommu_domain_destroy+0x16/0x18
>> <4>[ 5570.560030] [<ffffffff811fb5ea>] iommu_domain_free+0x16/0x22
>> <4>[ 5570.628611] [<ffffffffa0006261>] kvm_iommu_unmap_guest+0x22/0x28 [kvm]
>> <4>[ 5570.707570] [<ffffffffa0009b7b>] kvm_arch_destroy_vm+0x19/0x12a [kvm]
>> <4>[ 5570.785492] [<ffffffffa0002614>] kvm_put_kvm+0xe6/0x129 [kvm]
>> <4>[ 5570.855102] [<ffffffffa0002eb3>] kvm_vcpu_release+0x13/0x17 [kvm]
>> <4>[ 5570.928867] [<ffffffff8109cdfc>] fput+0x117/0x1be
>> <4>[ 5570.986013] [<ffffffff8109a147>] filp_close+0x63/0x6d
>> <4>[ 5571.047314] [<ffffffff810342dd>] put_files_struct+0x6f/0xda
>> <4>[ 5571.114845] [<ffffffff8103438e>] exit_files+0x46/0x4e
>> <4>[ 5571.176145] [<ffffffff81035b3d>] do_exit+0x1fc/0x681
>> <4>[ 5571.236416] [<ffffffffa000dedc>] ? kvm_arch_vcpu_ioctl_run+0xc2d/0xc55 [kvm]
>> <4>[ 5571.321605] [<ffffffff8138cc41>] ? __mutex_lock_slowpath+0x26c/0x294
>> <4>[ 5571.398490] [<ffffffff81036034>] do_group_exit+0x72/0x9a
>> <4>[ 5571.462907] [<ffffffff8103fec9>] get_signal_to_deliver+0x331/0x350
>> <4>[ 5571.537719] [<ffffffff81001f0f>] do_signal+0x6d/0x69a
>> <4>[ 5571.599013] [<ffffffff811da1fc>] ? put_ldisc+0x92/0x97
>> <4>[ 5571.661353] [<ffffffff810a95ea>] ? do_vfs_ioctl+0x527/0x576
>> <4>[ 5571.728887] [<ffffffff81002563>] do_notify_resume+0x27/0x51
>> <4>[ 5571.796419] [<ffffffff810a968c>] ? sys_ioctl+0x53/0x65
>> <4>[ 5571.858758] [<ffffffff81002b9b>] int_signal+0x12/0x17
>> <0>[ 5571.920058] Code: 48 85 d2 0f 95 c0 c9 c3 55 80 7f 4a 00 48 89 f8 48 89 e5 75 46 31 d2 48 8b 40 10 48 83 78 10 00 75 05 48 89 d0 eb 36 48 8b 40 38 <80> 78 4a 00 48 89 c2 74 e3 80 78 4b 07 74 23 80 3d 86 b5 5a 00
>> <1>[ 5572.145516] RIP [<ffffffff81197f8c>] pci_find_upstream_pcie_bridge+0x23/0x57
>> <4>[ 5572.230712] RSP <ffff8801f5f41a78>
>>
>> The two PCI devices in question are behind a PCIe bridge which is
>> connected to the rootport. The crash seems to be happening when
>> cleaning up the PCIe tree of the passed-through PCI devices. I tried
>> passing through the downstream ports of the bridge as well, but that
>> is not supported by qemu.
>>
>> Am I doing something wrong/unexpected here ? Any help in understanding
>> this issue will help me fix the issue properly.
>
> Please provide 'sudo lspci -vvv' from the host and the qemu commandline
> you're using. Is the bridge by chance an asmedia device? Thanks,
>
> Alex
>
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
