Hi,
I am running Ubuntu 10.10 (amd64) on a 2 socket nehalem based
server with IOH 5520. 5520 supports VTD.
I enabled DMAR with intel_iommu=on. The box has intel 82599 adapter
which I assigned through VT-D to FreeBSD 8.2 running
as guest os. The ixgbe driver detects the device and the driver
successfully configures the device. But the link
never comes up. It looks like link up/down interrupts are not
delivered. Then I checked kvm interrupt assignment and as expected
kvm could not make MSI-X entries for the VT-d guest. So no output from
"grep kvm /proc/interrupt". By enabling some debugs in the
qemu-kvm I figured out that the MSI-x updates are not received
properly. It does look like Linux updates MSI-X table in a batch
fashion
which qemu-kvm gets in one shot and every thing works fine in case of
linux. In case of FreeBSD PCIE updates come /MSI-X entry
which qemu-kvm can't make use.
To overcome this I compiled latest intel ixgbe driver with MSI-X
diabled. This time MSI interrupt got allocated both in the guest
and in the qemu-kvm (host kernel). Still I could not get link UP. I
modified the FreeBSD driver to poll for Link in some local_timer
task. Link comes up but no traffic flows. The MAC statistics show
packets received but packets do not reach the guest.
DMAing of packet may be failing. I could not find out the reason.
The same happens with legacy interrupt allocated in the guest. I even
tried qemu-kvm prefer_msi=off.
I think there are two problems1
1. Interrupt delivery either because of interrupt remapping failure or
all out interrupt allocation failure in qemu-kvm
2. Packets not getting DMAed to the guest possibly because of some DMAR issue.
Before doing VT-d I made sure connections are fine and the adapter
works fine in bare metal Linux.
I also tried VT-d of 82599 with Linux as guest (both 32 bits(PAE and
non-PAE) and amd64) and it just works magically.
Is this a known issue I am hitting with non-Linux guest oses and VT-d
? Appreciate any help in debugging this problem.
-Shashidhar
Linux kernel - 2.6.35
kvm - 0.14.1
CPU - Nehalem
IO chipset - 5520
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
