On Wed, Aug 01, 2012 at 04:36:14PM -0700, Yinghai Lu wrote: > > so it seems, that pic=nocsr is a must now. > > yes. Or you have bios provide SRIOV support or 64 bit resource in _CRS. Well, I can use PCI passthrough in Xen now, however, it seems SR-IOV does not work in case of Mellanox mlx4 driver. With 3.5 stock kernel, I got this message in virtual domain: [ 2.666623] mlx4_core: Mellanox ConnectX core driver v1.1 (Dec, 2011) [ 2.666635] mlx4_core: Initializing 0000:00:00.1 [ 2.666717] mlx4_core 0000:00:00.1: enabling device (0000 -> 0002) [ 2.666975] mlx4_core 0000:00:00.1: Xen PCI mapped GSI0 to IRQ168 [ 2.667040] mlx4_core 0000:00:00.1: enabling bus mastering [ 2.667184] mlx4_core 0000:00:00.1: Detected virtual function - running in slave mode [ 2.667214] mlx4_core 0000:00:00.1: Sending reset [ 2.667319] mlx4_core 0000:00:00.1: Sending vhcr0 [ 2.667886] mlx4_core 0000:00:00.1: HCA minimum page size:1 [ 2.668067] mlx4_core 0000:00:00.1: The host doesn't support eth interface [ 2.668074] mlx4_core 0000:00:00.1: QUERY_FUNC_CAP command failed, aborting. [ 2.668079] mlx4_core 0000:00:00.1: Failed to obtain slave caps [ 2.668305] mlx4_core: probe of 0000:00:00.1 failed with error -93 not sure what does it mean. I also tried OFED package from Mellanox which seems to have better SR-IOV support (at least mlx4_ib does not complain that SR-IOV is not supported). However, it does not work when SR-IOV enabled: [13677.034266] mlx4_core 0000:02:00.0: Running in master mode [13689.278238] mlx4_core 0000:02:00.0: command 0x31 timed out (go bit not cleared) [13689.278324] mlx4_core 0000:02:00.0: NOP command failed to generate MSI-X interrupt IRQ 241). [13689.278399] mlx4_core 0000:02:00.0: Trying again without MSI-X. [13699.286473] mlx4_core 0000:02:00.0: command 0x31 timed out (go bit not cleared) [13699.286557] mlx4_core 0000:02:00.0: NOP command failed to generate interrupt (IRQ 237), aborting. [13699.286633] mlx4_core 0000:02:00.0: BIOS or ACPI interrupt routing problem? [13701.406680] mlx4_core: probe of 0000:02:00.0 failed with error -16 if I disable SR-IOV mode for this driver, it works OK. Could the interrupt problem be BIOS related? I.e., it won't work until I got BIOS which properly supports SR-IOV with Mellanox card? -- Lukáš Hejtmánek -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
