On Wednesday 08 April 2009 00:38:10 Alex Williamson wrote:
> On Tue, 2009-04-07 at 14:09 +0800, Sheng Yang wrote:
> > On Saturday 04 April 2009 05:27:43 Alex Williamson wrote:
> > > Do we need some disable logic here? If I toggle a bnx2 NIC in a guest,
> > > I get the following when it attempts to come back up:
> > >
> > > MSI-X entry number is zero!
> > > assigned_dev_update_msix_mmio: No such device or address
> >
> > It seems that driver didn't fill the MMIO with any correct MSIX
> > information, or the program fail to intercept it after driver set enable
> > bit of MSIX. It's strange... (Have it got something to do with PM and
> > some EXP feature you mentioned?)
>
> My guess was that it filled in the MSIX info, but then can't find a free
> slot to reload the MSIX data when it tries to re-enable MSIX. I hacked
> the bnx2 driver to not rely on PM and EXP capabilities for this test, it
> seems to work, but it's possible that I broke something. My host also
> locks up the second time I try to export this device to a guest, maybe a
> problem with my bnx2 hacks, MSIX not getting reset, or prototype
> hardware. I'll see if I can find another MSIX capable device to export
> to a guest.
>
> > Could you enable DEVICE_ASSSIGNMENT_DEBUG=1 in
> > qemu/hw/device-assignment.c and post the output?
>
> Yup, see below. The error comes after I 'ifdown eth0; ifup eth0' in the
> guest. Note bnx2 appears to only turn on MSIX for SMP systems. Thanks,
>
> Alex
Seems your "ifdown/ifup" script reload the module? Oh god, I found one bug
after checked the spec:
System software reads this field to determine the MSI-X Table Size *N*, which
is encoded as *N-1*. For example, a returned value of “00000000011” indicates
a table size of 4.
But it seems still can't explain the problem...(OK, it may affect the guest in
a unknown way as well...) I would post a fix for it soon.
> val=0x00000008 len=2 assigned_dev_pci_write_config: (4.0): address=0052
> val=0x00000008 len=2 assigned_dev_pci_read_config: (4.0): address=0006
> val=0x00000010 len=2 assigned_dev_pci_read_config: (4.0): address=0034
> val=0x00000040 len=1 assigned_dev_pci_read_config: (4.0): address=0040
> val=0x00000005 len=1 assigned_dev_pci_read_config: (4.0): address=0041
> val=0x00000050 len=1 assigned_dev_pci_read_config: (4.0): address=0050
> val=0x00000011 len=1 assigned_dev_pci_read_config: (4.0): address=0052
> val=0x00000008 len=2 assigned_dev_pci_read_config: (4.0): address=0054
> val=0x0000c000 len=4 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x0, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table
> mmio offset 0x4, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x8, val 0x4191 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x10, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table
> mmio offset 0x14, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x18, val 0x4199 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x20, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table
> mmio offset 0x24, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x28, val 0x41a1 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x30, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table
> mmio offset 0x34, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x38, val 0x41a9 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x40, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table
> mmio offset 0x44, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x48, val 0x41b1 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x50, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table
> mmio offset 0x54, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x58, val 0x41b9 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x60, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table
> mmio offset 0x64, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x68, val 0x41c1 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x70, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table
> mmio offset 0x74, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x78, val 0x41c9 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x80, val 0xfeeff00c msix_mmio_writel: write to MSI-X entry table
> mmio offset 0x84, val 0x0 msix_mmio_writel: write to MSI-X entry table mmio
> offset 0x88, val 0x41d1 assigned_dev_pci_read_config: (4.0): address=0004
> val=0x00000046 len=2 assigned_dev_pci_write_config: (4.0): address=0004
> val=0x00000446 len=2 assigned_dev_pci_write_config: NON BAR (4.0):
The writing to MMIO have been intercepted, but code fail to count it?
Strange...
Could you try this debug?
diff --git a/qemu/hw/device-assignment.c b/qemu/hw/device-assignment.c
index 09e54ae..ba31bed 100644
--- a/qemu/hw/device-assignment.c
+++ b/qemu/hw/device-assignment.c
@@ -45,7 +45,7 @@
#define IORESOURCE_DMA 0x00000800
#define IORESOURCE_PREFETCH 0x00001000 /* No side effects */
-/* #define DEVICE_ASSIGNMENT_DEBUG 1 */
+#define DEVICE_ASSIGNMENT_DEBUG 1
#ifdef DEVICE_ASSIGNMENT_DEBUG
#define DEBUG(fmt, ...) \
@@ -816,9 +816,13 @@ static int assigned_dev_update_msix_mmio(PCIDevice
*pci_dev)
else
pos = pci_dev->cap.start;
+ fprintf(stderr, "the MSIX capabilty position is 0x%lx\n", pos);
+
entries_max_nr = pci_dev->config[pos + 2];
entries_max_nr &= PCI_MSIX_TABSIZE;
+ fprintf(stderr, "the MSIX entries_max_nr is 0x%lx\n", entries_max_nr);
+
/* Get the usable entry number for allocating */
for (i = 0; i < entries_max_nr; i++) {
memcpy(&msg_ctrl, va + i * 16 + 12, 4);
(Further more, you can check the content of msg_ctrl when code try to count
them). And also please try the fix for MSI-X table size.
Thanks!
--
regards
Yang, Sheng
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
