+ Bairen Thanks -----Original Message----- From: Leon Romanovsky [mailto:leon@xxxxxxxxxx] Sent: Monday, November 20, 2017 3:24 PM To: Junxue ZHANG <jzhangcs@xxxxxxxxxxxxxx>; Saeed Mahameed <saeedm@xxxxxxxxxxxx> Cc: linux-rdma@xxxxxxxxxxxxxxx; Or Gerlitz <ogerlitz@xxxxxxxxxxxx> Subject: Re: Problem of changing the inline mode or turning on the encapsulation support of Mellanox NIC’s eswitch On Mon, Nov 20, 2017 at 03:29:00AM +0000, Junxue ZHANG wrote: > Hi all, > > I encountered a problem when I tried to change the inline mode or turn on the encapsulation support of Mellanox NIC’s eswitch. I wonder if anyone could help me with this. Thanks. > > I want to use VXLan with OVS and try to offload the rules in hardware. I don’t know whether it is necessary to turn on the encapsulation support. It would also be of great help if anyone could tell me the correct way to accomplish that. Thanks. > + Saeed and Or The failure in drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c 990 err = esw_inline_mode_from_devlink(mode, &mlx5_mode); 991 if (err) 992 goto out; 993 994 for (vport = 1; vport < esw->enabled_vports; vport++) { 995 err = mlx5_modify_nic_vport_min_inline(dev, vport, mlx5_mode); 996 if (err) { 997 esw_warn(dev, "Failed to set min inline on vport %d\n", 998 vport); 999 goto revert_inline_mode; 1000 } 1001 } Thanks > > Problem: > ————————— > I can successfully turned on the switchdev mode of my Mellanox NIC. But when I tried to change the inline-mode or turn on the encapsulation support, it failed. > > I can create VFs successfully and use those VFs with switch with link inline mode and encapsulation support off. > > How to reproduce: > ————————— > Before each of the following steps, I first turn the switchdev mode off and turn on the legacy mode. > > 1. Change the inline mode: > > $ devlink dev eswitch set pci/0000:81:00.0 mode switchdev inline-mode > network devlink answers: Invalid argument > > Output of Syslog: > Nov 20 00:05:04 ip-192-168-2-203 kernel: [ 2968.592023] > mlx5_0:wait_for_async_commands:735:(pid 2332): done with all pending > requests Nov 20 00:05:04 ip-192-168-2-203 systemd-udevd[2335]: Could > not generate persistent MAC address for eth0: No such file or > directory Nov 20 00:05:04 ip-192-168-2-203 systemd-udevd[2340]: Could > not generate persistent MAC address for eth1: No such file or > directory Nov 20 00:05:04 ip-192-168-2-203 kernel: [ 2968.707239] > (0000:81:00.0): E-Switch: SRIOV enabled: active vports(3) Nov 20 > 00:05:04 ip-192-168-2-203 kernel: [ 2968.707712] mlx5_core > 0000:81:00.0: mlx5_cmd_check:710:(pid 2332): > MODIFY_NIC_VPORT_CONTEXT(0x755) op_mod(0x0) failed, status bad > parameter(0x3), syndrome (0x40ea03) Nov 20 00:05:04 ip-192-168-2-203 > kernel: [ 2968.707790] (0000:81:00.0): E-Switch: Failed to set min > inline on vport 1 > > 2. Turn on the encapsulation: > > $ devlink dev eswitch set pci/0000:81:00.0 mode switchdev encap enable > devlink answers: Operation not supported > > Output of Syslog: > > Nov 20 00:06:16 ip-192-168-2-203 kernel: [ 3040.121748] > (0000:81:00.0): E-Switch: disable SRIOV: active vports(3) mode(1) Nov > 20 00:06:16 ip-192-168-2-203 kernel: [ 3040.161942] (0000:81:00.0): > E-Switch: E-Switch enable SRIOV: nvfs(2) mode (2) Nov 20 00:06:18 > ip-192-168-2-203 kernel: [ 3042.847377] > mlx5_0:wait_for_async_commands:735:(pid 2371): done with all pending > requests Nov 20 00:06:19 ip-192-168-2-203 systemd-udevd[2374]: Could > not generate persistent MAC address for eth0: No such file or > directory Nov 20 00:06:19 ip-192-168-2-203 systemd-udevd[2379]: Could > not generate persistent MAC address for eth1: No such file or > directory Nov 20 00:06:19 ip-192-168-2-203 kernel: [ 3042.961906] > (0000:81:00.0): E-Switch: SRIOV enabled: active vports(3) > > 3. Query the status > > $ devlink dev eswitch show pci/0000:81:00.0 > pci/0000:81:00.0: mode switchdev inline-mode link encap disable > > > NIC hardware model: > ————————— > ConnectX®-4 EN network interface card, 40GbE dual-port QSFP28, PCIe3.0 > x8, tall bracket, ROHS R6 > > > NIC firmware: > ————————— > $ mstflint -d 81:00.1 q > > Image type: FS3 > FW Version: 12.21.1000 > FW Release Date: 29.10.2017 > Product Version: rel-12_21_1000 > Rom Info: type=PXE version=3.5.305 devid=4115 cpu=AMD64 > Description: UID GuidsNumber Base GUID: > ec0d9a0300373802 4 Base MAC: 0000ec0d9a373802 > 4 Image VSD: N/A Device VSD: N/A > PSID: MT_2130110027 > Security Attributes: N/A > > Linux distribution: > ————————— > $ cat /etc/*-release > > PRETTY_NAME="Debian GNU/Linux 9 (stretch)" > NAME="Debian GNU/Linux" > VERSION_ID="9" > VERSION="9 (stretch)" > ID=debian > HOME_URL="https://www.debian.org/"; > SUPPORT_URL="https://www.debian.org/support"; > BUG_REPORT_URL="https://bugs.debian.org/"; > > Linux kernel: > ————————— > $ uname -r > > 4.14.0-rc7-amd64 > > NIC driver: > ————————— > $ ethtool -i ens6f0 > > driver: mlx5_core > version: 5.0-0 > firmware-version: 12.21.1000 (MT_2130110027) > expansion-rom-version: > bus-info: 0000:81:00.0 > supports-statistics: yes > supports-test: yes > supports-eeprom-access: no > supports-register-dump: no > supports-priv-flags: yes > > Best, > > Junxue > > -- > To unsubscribe from this list: send the line "unsubscribe linux-rdma" > in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo > info at http://vger.kernel.org/majordomo-info.html ��.n��������+%������w��{.n�����{���fk��ܨ}���Ơz�j:+v�����w����ޙ��&�)ߡ�a����z�ޗ���ݢj��w�f
