On 06.05.24 06:46, shaozhengchao wrote:
When using the 5.10 kernel, I can find two IB devices using the
ibv_devinfo command.
----------------------------------
[root@localhost ~]# lspci
91:00.0 Ethernet controller: Mellanox Technologies MT27800 Family
[ConnectX-5]
91:00.1 Ethernet controller: Mellanox Technologies MT27800 Family
----------------------------------
[root@localhost ~]# ibv_devinfo
hca_id: mlx5_0
transport: InfiniBand (0)
fw_ver: 16.31.1014
node_guid: f41d:6b03:006f:4743
sys_image_guid: f41d:6b03:006f:4743
vendor_id: 0x02c9
vendor_part_id: 4119
hw_ver: 0x0
board_id: HUA0000000004
phys_port_cnt: 1
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 1024 (3)
sm_lid: 0
port_lid: 0
port_lmc: 0x00
link_layer: Ethernet
hca_id: mlx5_1
transport: InfiniBand (0)
fw_ver: 16.31.1014
node_guid: f41d:6b03:006f:4744
sys_image_guid: f41d:6b03:006f:4743
vendor_id: 0x02c9
vendor_part_id: 4119
hw_ver: 0x0
board_id: HUA0000000004
phys_port_cnt: 1
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 1024 (3)
sm_lid: 0
port_lid: 0
port_lmc: 0x00
link_layer: Ethernet
----------------------------------
But after the two network ports are bonded, only one IB device is
available, and only PF0 can be used.
[root@localhost shaozhengchao]# ibv_devinfo
hca_id: mlx5_bond_0
transport: InfiniBand (0)
fw_ver: 16.31.1014
node_guid: f41d:6b03:006f:4743
sys_image_guid: f41d:6b03:006f:4743
vendor_id: 0x02c9
vendor_part_id: 4119
hw_ver: 0x0
board_id: HUA0000000004
phys_port_cnt: 1
port: 1
state: PORT_ACTIVE (4)
max_mtu: 4096 (5)
active_mtu: 1024 (3)
sm_lid: 0
port_lid: 0
port_lmc: 0x00
link_layer: Ethernet
The current Linux mainline driver is the same.
I found the comment ("If bonded, we do not add an IB device for PF1.")
in the mlx5_lag_intf_add function of the 5.10 branch driver code.
Not sure if rdma lag is enabled for this or not. /proc/net/bonding will
provide more more details normally.
Zhu Yanjun
This indicates that wthe the same NIC is used, only PF0 support bonding?
Are there any other constraints, when enable bonding with CX5?
Thank you
Zhengchao Shao
