If IOMMU support 64-bit coherent DMA mlx4_core driver will try to use it
and VF probe will fail due to firmware error.
Force all mlx4_core VFs coherent DMA to 32-bit only!
Kernel log when issue occur:
[1383654.766249] mlx4_core 0006:01:00.1: Sending reset
[1383654.775971] mlx4_core 0006:01:00.0: Received reset from slave:1
[1383654.788087] mlx4_core 0006:01:00.1: Sending vhcr0
[1383664.318338] mlx4_core 0006:01:00.0: command 0x2e failed: fw status = 0x1
[1383664.318342] mlx4_core 0006:01:00.0: mlx4_master_process_vhcr: Failed
reading vhcr ret: 0xfffffffb
[1383664.318345] mlx4_core 0006:01:00.0: Failed processing vhcr for slave:1,
resetting slave
[1383664.318352] mlx4_core 0006:01:00.0: Turn on internal error to force
reset, slave=1, cmd=0x5
[1383664.318415] mlx4_core 0006:01:00.0: slave:1 is out of sync, cmd=0x5,
last command=0x0, reset is needed
[1383664.318418] mlx4_core 0006:01:00.0: Turn on internal error to force
reset, slave=1, cmd=0x5
[1383664.318501] mlx4_core 0006:01:00.0: slave:1 is out of sync, cmd=0x5,
last command=0x0, reset is needed
[1383664.318504] mlx4_core 0006:01:00.0: Turn on internal error to force
reset, slave=1, cmd=0x5
[1383664.318513] mlx4_core 0006:01:00.1: HCA minimum page size:1
[1383664.318515] mlx4_core 0006:01:00.1: UAR size:4096 != kernel PAGE_SIZE of
8192
[1383664.318517] mlx4_core 0006:01:00.1: Failed to obtain slave caps
Signed-off-by: Shamir Rabinovitch <shamir.rabinovitch@xxxxxxxxxx>
---
Changelog:
v1 -> v2:
Review comments from Christoph Hellwig <hch@xxxxxxxxxxxxx>.
Verified and only VF require 32-bit coherent DMA.
PF can still use 64-bit coherent DMA.
---
---
drivers/net/ethernet/mellanox/mlx4/main.c | 20 +++++++++++++++++---
1 files changed, 17 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
index bffa6f3..131cbc9 100644
--- a/drivers/net/ethernet/mellanox/mlx4/main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/main.c
@@ -3719,9 +3719,23 @@ static int __mlx4_init_one(struct pci_dev *pdev, int pci_dev_data,
goto err_release_regions;
}
}
- err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
- if (err) {
- dev_warn(&pdev->dev, "Warning: couldn't set 64-bit consistent PCI DMA mask\n");
+ if (!(pci_dev_data & MLX4_PCI_DEV_IS_VF)) {
+ err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ if (err) {
+ dev_warn(&pdev->dev,
+ "Warning: couldn't set 64-bit consistent PCI DMA mask\n");
+ err = pci_set_consistent_dma_mask(pdev,
+ DMA_BIT_MASK(32));
+ if (err) {
+ dev_err(&pdev->dev,
+ "Can't set consistent PCI DMA mask, aborting\n");
+ goto err_release_regions;
+ }
+ }
+ } else {
+ /* CX3 firmware 2.11.1280 does not support 64-bit coherent
+ * DMA for VFs.
+ */
err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
if (err) {
dev_err(&pdev->dev, "Can't set consistent PCI DMA mask, aborting\n");
--
1.7.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
