On Fri, 2012-01-13 at 22:33 +0100, Jan Kiszka wrote: > On 2012-01-13 22:05, Alex Williamson wrote: > > On Fri, 2012-01-13 at 22:00 +0100, Jan Kiszka wrote: > >> On 2012-01-04 04:21, Alex Williamson wrote: > >>> On Mon, 2011-12-19 at 19:49 +0530, Shashidhar Patil wrote: > >>>> Hi, > >>>> I am running Ubuntu 10.10 (amd64) on a 2 socket nehalem based > >>>> server with IOH 5520. 5520 supports VTD. > >>>> I enabled DMAR with intel_iommu=on. The box has intel 82599 adapter > >>>> which I assigned through VT-D to FreeBSD 8.2 running > >>>> as guest os. The ixgbe driver detects the device and the driver > >>>> successfully configures the device. But the link > >>>> never comes up. It looks like link up/down interrupts are not > >>>> delivered. Then I checked kvm interrupt assignment and as expected > >>>> kvm could not make MSI-X entries for the VT-d guest. So no output from > >>>> "grep kvm /proc/interrupt". By enabling some debugs in the > >>>> qemu-kvm I figured out that the MSI-x updates are not received > >>>> properly. It does look like Linux updates MSI-X table in a batch > >>>> fashion > >>>> which qemu-kvm gets in one shot and every thing works fine in case of > >>>> linux. In case of FreeBSD PCIE updates come /MSI-X entry > >>>> which qemu-kvm can't make use. > >>> > >>> That's right, Linux and Windows both seem to setup the MSI-X table then > >>> enable it in one shot, so we only trigger the interrupt programming when > >>> the enable bit is set. We don't trigger changes on writes to the MSI-X > >>> table... not very accurate emulation of mask bits. > >> > >> According to the PCI spec, updates that happen while a vector is > >> unmasked, need not be considered by the hardware (thus the hypervisor > >> here). Is that the scenario here? > > > > I'm assuming the vector is masked in the MSI-X table. So Linux/Windows > > do: > > > > a) program MSI-X table > > b) enable MSI-X in capability register > > > > Whereas FreeBSD does: > > > > a) enable MSI-X in capability register (vectors masked in table) > > b) program and unmask individual vectors > > That should work with the current code. It checks the number of vectors > on each config write, iterates the whole table, and then updates the ^^^^^^^^^^^^^^^^^^^^ > kernel configuration accordingly. It even requires the enable bit in the > cap register to be set before doing this. That's the problem, we only do it on config writes overlapping the MSI-X flags. We don't do anything for writes to the MSI-X table. It might be as simple as calling assigned_dev_update_msix() from msix_mmio_writel() when the mask bit is toggled. I'm not sure what might fall out of that though. Thanks, Alex -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
