Re: SR-IOV: IGB/IGBVF initialisation problem

[Yu Zhao <yu.zhao@xxxxxxxxx>]

Your BIOS doesn't support SR-IOV. Please contact the vendor for an 
up-to-date one.

Using pci=assign-busses also can solve the problem, but it's not 
recommended since BIOS is responsible for reserving bus for VFs.

Simon Horman wrote:
> Sorry, it was me who sent the previous mail, not Jesse Barnes.
> I didn't intend to pretend to be him :-(
>
> On Thu, Jul 16, 2009 at 06:30:26PM +1000, Jesse Barnes wrote:
> > Hi,
> >
> > I am having some trouble initialising the virtual functions
> > of an intel 85276 NIC.
> >
> > The first problem that  I see is that if I insert the igb
> > module with max_vfs > 0 is the following:
> >
> >     igb 0000:01:00.1: SR-IOV: bus number out of range
> >
> > I added some debugging messages to the relevant code
> > and it seems that iov wants to add the vf to bus 2,
> > but bus->subordiante is set to 1.
> >
> >     virtfn_bus: dev->bus->number:   0x1
> >     virtfn_bus: dev->devfn:         0x1
> >     virtfn_bus: dev->sriov->offset: 0x180
> >     virtfn_bus: dev->sriov->stride: 0x2
> >     virtfn_bus: id:                 0x0
> >     virtfn_bus: result:             0x2
> >
> > If I disable this check (probably unwise, but I wanted to see what would
> > happen), then the virtual function is indeed added on bus 2, and my
> > lspci looks a bit like this:
> >
> >     01:00.0 Ethernet controller: Intel Corporation Device 10c9 (rev 01)
> >     01:00.1 Ethernet controller: Intel Corporation Device 10c9 (rev 01)
> >     02:10.0 Class ffff: Intel Corporation Device 10ca (rev ff)
> >     02:10.1 Class ffff: Intel Corporation Device 10ca (rev ff)
> >
> > That is, each of the PFs (01:00.0 and 01:00.1) end up with one VF each.
> > I ran this test with max_vfs=1.
> >
> > However, when I try to bring up one of these VFs using ifconfig,
> > the kernel panics, as per the log below.
> >
> > Is this a know problem? Is there something peculiar about my hw,
> > its an Intel 85276 plugged into an HP dc7800? Am I doing something wrong?
> > U
> >
> >
> > [  135.976056] igb 0000:01:00.1: PCI INT B disabled
> > [  136.040051] igb 0000:01:00.0: PCI INT A disabled
> > [  136.104935] Intel(R) Gigabit Ethernet Network Driver - version 1.3.16-k2
> > [  136.111617] Copyright (c) 2007-2009 Intel Corporation.
> > [  136.116784] igb 0000:01:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
> > [  136.123674] virtfn_bus: parent:             sriov_enable
> > [  136.128971] virtfn_bus: dev->bus->number:   0x1
> > [  136.133489] virtfn_bus: dev->devfn:         0x0
> > [  136.138006] virtfn_bus: dev->sriov->stride: 0x2
> > [  136.142521] virtfn_bus: id:                 0x0
> > [  136.147038] virtfn_bus: result:             0x2
> > [  136.151555] igb 0000:01:00.0: SR-IOV: nr_virtfn: 1
> > [  136.156332] virtfn_bus: parent:             sriov_enable
> > [  136.161629] virtfn_bus: dev->bus->number:   0x1
> > [  136.166146] virtfn_bus: dev->devfn:         0x0
> > [  136.170663] virtfn_bus: dev->sriov->stride: 0x2
> > [  136.175181] virtfn_bus: id:                 0x0
> > [  136.179697] virtfn_bus: result:             0x2
> > [  136.184214] igb 0000:01:00.0: SR-IOV: virtfn_bus(dev, nr_virtfn - 1): 2
> > [  136.190811] igb 0000:01:00.0: SR-IOV: dev->bus->subordinate: 1
> > [  136.196628] igb 0000:01:00.0: SR-IOV: bus number out of range
> > [  136.304018] virtfn_bus: parent:             virtfn_add
> > [  136.309142] virtfn_bus: dev->bus->number:   0x1
> > [  136.313658] virtfn_bus: dev->devfn:         0x0
> > [  136.318175] virtfn_bus: dev->sriov->stride: 0x2
> > [  136.322693] virtfn_bus: id:                 0x0
> > [  136.327210] virtfn_bus: result:             0x2
> > [  136.331745] virtfn_add: 0x180bd000
> > [  136.335146] pci 0000:02:10.0: unknown header type 7f, ignoring device
> > [  136.341643] igb 0000:01:00.0: 1 vfs allocated
> > [  136.386346] Intel(R) Virtual Function Network Driver - version 1.0.0-k0
> > [  136.392943] Copyright (c) 2009 Intel Corporation.
> > [  136.534817] igb 0000:01:00.0: Intel(R) Gigabit Ethernet Network Connection
> > [  136.541675] igb 0000:01:00.0: eth3: (PCIe:2.5Gb/s:Width x4) 00:1b:21:2f:16:84
> > [  136.548860] igb 0000:01:00.0: eth3: PBA No: e43709-003
> > [  136.553984] igb 0000:01:00.0: Using MSI-X interrupts. 2 rx queue(s), 2 tx queue(s)
> > [  136.561582] igbvf 0000:02:10.0: Failed to initialize MSI-X interrupts.
> > [  136.569230] igbvf 0000:02:10.0: PF still in reset state, assigning new address
> > [  136.577571] igbvf 0000:02:10.0: PF still resetting
> > [  136.582807] igbvf 0000:02:10.0: Intel(R) 82576 Virtual Function
> > [  136.588713] igbvf 0000:02:10.0: Address: 56:3a:48:ef:a5:52
> > [  136.594181] igbvf 0000:02:10.0: MAC: 1
> > [  136.598005] igb 0000:01:00.1: PCI INT B -> GSI 17 (level, low) -> IRQ 17
> > [  136.604893] virtfn_bus: parent:             sriov_enable
> > [  136.610188] virtfn_bus: dev->bus->number:   0x1
> > [  136.614703] virtfn_bus: dev->devfn:         0x1
> > [  136.619219] virtfn_bus: dev->sriov->stride: 0x2
> > [  136.623735] virtfn_bus: id:                 0x0
> > [  136.628253] virtfn_bus: result:             0x2
> > [  136.632770] igb 0000:01:00.1: SR-IOV: nr_virtfn: 1
> > [  136.637547] virtfn_bus: parent:             sriov_enable
> > [  136.642844] virtfn_bus: dev->bus->number:   0x1
> > [  136.647360] virtfn_bus: dev->devfn:         0x1
> > [  136.651878] virtfn_bus: dev->sriov->stride: 0x2
> > [  136.656396] virtfn_bus: id:                 0x0
> > [  136.660914] virtfn_bus: result:             0x2
> > [  136.665429] igb 0000:01:00.1: SR-IOV: virtfn_bus(dev, nr_virtfn - 1): 2
> > [  136.672024] igb 0000:01:00.1: SR-IOV: dev->bus->subordinate: 1
> > [  136.677839] igb 0000:01:00.1: SR-IOV: bus number out of range
> > [  136.784019] virtfn_bus: parent:             virtfn_add
> > [  136.789140] virtfn_bus: dev->bus->number:   0x1
> > [  136.793656] virtfn_bus: dev->devfn:         0x1
> > [  136.798173] virtfn_bus: dev->sriov->stride: 0x2
> > [  136.802691] virtfn_bus: id:                 0x0
> > [  136.807206] virtfn_bus: result:             0x2
> > [  136.811725] virtfn_add: 0x180bd000
> > [  136.815123] pci 0000:02:10.1: unknown header type 7f, ignoring device
> > [  136.821652] igbvf 0000:02:10.1: Failed to initialize MSI-X interrupts.
> > [  136.829299] igbvf 0000:02:10.1: PF still in reset state, assigning new address
> > [  136.837640] igbvf 0000:02:10.1: PF still resetting
> > [  136.842937] igbvf 0000:02:10.1: Intel(R) 82576 Virtual Function
> > [  136.848841] igbvf 0000:02:10.1: Address: 62:00:68:2d:2b:10
> > [  136.854311] igbvf 0000:02:10.1: MAC: 1
> > [  136.858143] igb 0000:01:00.1: 1 vfs allocated
> > [  137.046896] igb 0000:01:00.1: Intel(R) Gigabit Ethernet Network Connection
> > [  137.053756] igb 0000:01:00.1: eth5_rename: (PCIe:2.5Gb/s:Width x4) 00:1b:21:2f:16:85
> > [  137.061546] igb 0000:01:00.1: eth5_rename: PBA No: e43709-003
> > [  137.067272] igb 0000:01:00.1: Using MSI-X interrupts. 2 rx queue(s), 2 tx queue(s)
> > [  269.490297] BUG: unable to handle kernel NULL pointer dereference at 00000000000005d1
> > [  269.493873] IP: [<ffffffff811789ad>] pci_find_upstream_pcie_bridge+0x23/0x60
> > [  269.493873] PGD 11985f067 PUD 1194f2067 PMD 0 
> > [  269.493873] Oops: 0000 [#1] SMP 
> > [  269.493873] last sysfs file: /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.1/net/eth5/type
> > [  269.493873] CPU 0 
> > [  269.493873] Modules linked in: igbvf igb ipv6 loop snd_hda_codec_analog snd_hda_intel snd_hda_codec iTCO_wdt evdev psmouse pcspkr serio_raw video output wmi parport_pc parport snd_hwdep snd_pcm snd_timer snd soundcore snd_page_alloc tpm_infineon tpm tpm_bios button processor intel_agp dm_mirror dm_region_hash dm_log dm_snapshot dm_mod sg sr_mod cdrom sd_mod crc_t10dif ata_piix ata_generic libata scsi_mod e100 mii e1000 ehci_hcd ide_pci_generic uhci_hcd e1000e ide_core dca floppy thermal fan thermal_sys ext2 ext3 jbd mbcache [last unloaded: igb]
> > [  269.493873] Pid: 3343, comm: ifconfig Not tainted 2.6.31-rc3 #14 HP Compaq dc7800p Convertible Minitower
> > [  269.493873] RIP: 0010:[<ffffffff811789ad>]  [<ffffffff811789ad>] pci_find_upstream_pcie_bridge+0x23/0x60
> > [  269.493873] RSP: 0018:ffff880118c93c28  EFLAGS: 00010286
> > [  269.493873] RAX: ffff8801180bd000 RBX: ffff88011816c000 RCX: 0000000000000000
> > [  269.493873] RDX: 0000000000000000 RSI: 0000000000000030 RDI: 0000000000000000
> > [  269.493873] RBP: 00000001168c0000 R08: ffffffffffffffff R09: ffff88011bc45640
> > [  269.493873] R10: ffff880000010d08 R11: 0000000200000000 R12: 0000000000000030
> > [  269.493873] R13: ffff88011816c000 R14: ffff88011816c080 R15: 0000000000000000
> > [  269.493873] FS:  00007f95b6d2c6e0(0000) GS:ffff88002801f000(0000) knlGS:0000000000000000
> > [  269.493873] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> > [  269.493873] CR2: 00000000000005d1 CR3: 00000001194df000 CR4: 00000000000006f0
> > [  269.493873] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > [  269.493873] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> > [  269.493873] Process ifconfig (pid: 3343, threadinfo ffff880118c92000, task ffff8801195454c0)
> > [  269.493873] Stack:
> > [  269.493873]  ffff88000000fc00 ffffffff81181d34 ffffc9000000cfff ffffffff810bf485
> > [  269.493873] <0> 00000000ffffffff 0000000000000000 ffff880000012800 ffff88011816c000
> > [  269.493873] <0> 00000001168c0000 0000000000004000 0000000000000002 ffff88011816c080
> > [  269.493873] Call Trace:
> > [  269.493873]  [<ffffffff81181d34>] ? get_domain_for_dev+0x46/0x47a
> > [  269.493873]  [<ffffffff810bf485>] ? vmap_page_range_noflush+0x287/0x2ec
> > [  269.493873]  [<ffffffff8118217b>] ? get_valid_domain_for_dev+0x13/0x6d
> > [  269.493873]  [<ffffffff81182925>] ? __intel_map_single+0x41/0x14b
> > [  269.493873]  [<ffffffff81182dfd>] ? intel_alloc_coherent+0x78/0x9d
> > [  269.493873]  [<ffffffffa0347e12>] ? igbvf_setup_tx_resources+0x5d/0xbc [igbvf]
> > [  269.493873]  [<ffffffffa0347ea0>] ? igbvf_open+0x2f/0x12b [igbvf]
> > [  269.493873]  [<ffffffff8122e609>] ? dev_open+0xa4/0xe6
> > [  269.493873]  [<ffffffff8122dd83>] ? dev_change_flags+0xaf/0x16d
> > [  269.493873]  [<ffffffff81273bf3>] ? devinet_ioctl+0x264/0x509
> > [  269.493873]  [<ffffffff8121e278>] ? sock_ioctl+0x1e7/0x20b
> > [  269.493873]  [<ffffffff810dbe06>] ? vfs_ioctl+0x21/0x6c
> > [  269.493873]  [<ffffffff810dc2fe>] ? do_vfs_ioctl+0x437/0x475
> > [  269.493873]  [<ffffffff8102f388>] ? do_page_fault+0x1fa/0x20f
> > [  269.493873]  [<ffffffff810dc38d>] ? sys_ioctl+0x51/0x70
> > [  269.493873]  [<ffffffff81010a02>] ? system_call_fastpath+0x16/0x1b
> > [  269.493873] Code: c0 0f b6 c0 c3 31 c0 c3 48 83 ec 08 f6 87 d1 05 00 00 20 75 4c 31 d2 48 8b 47 10 48 83 78 10 00 75 05 48 89 d7 eb 3c 48 8b 78 38 <f6> 87 d1 05 00 00 20 48 89 fa 74 e0 80 7f 4a 07 74 26 83 3d ca 
> > [  269.493873] RIP  [<ffffffff811789ad>] pci_find_upstream_pcie_bridge+0x23/0x60
> > [  269.493873]  RSP <ffff880118c93c28>
> > [  269.493873] CR2: 00000000000005d1
> > [  269.830497] ---[ end trace 03cd77c3528343dd ]---

--
To unsubscribe from this list: send the line "unsubscribe linux-pci" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html