On Mon, 14 Dec 2020 14:46:30 +0100
Patrick Bégou <Patrick.Begou@xxxxxxxxxxxxxxxxxxxx> wrote:
> Hi,
>
> I'm deploying a CentOS8 (not stream 😁) cluster and I have a question
> about MTU on the interfaces. I have a connectX6 Mellanox interface
> where I need IBoIP setup.
> I've setup this interface via nmcli and set the MTU to 65520 with:
>
> nmcli connection modify ib0 mtu 65520
> nmcli connection up ib0
>
> Running "nmcli connection show ib0" report:
>
> infiniband.mtu: 65520
>
> But "ip addr show ib0" report a mtu of 2044:
>
> 6: ib0: <BROADCAST,MULTICAST,UP,LOWER_UP> *mtu 2044 *qdisc mq
> state UP group default qlen 256
2044 is what you get with DATAGRAM mode and the native IB running with
2K packages. You can get 4092 IPoIB MTU with 4K IB. And you do want to
run 4K native IB in general.
To run 64K MTU you need to switch from DATAGRAM mode to CONNECTED mode
for IPoIB. CONNECTED mode give the highest single stream bandwidth
typically but may not give best overall performance (vs 4092 DATAGRAM
mode).
Cheers,
Peter K
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos
