On Fri, Jan 17, 2020 at 6:16 PM Ryan Goodfellow <rgoodfel@xxxxxxx> wrote: > > On Fri, Jan 17, 2020 at 01:32:07PM +0100, Björn Töpel wrote: > > On Mon, 13 Jan 2020 at 01:28, Ryan Goodfellow <rgoodfel@xxxxxxx> wrote: > > > > > [...] > > > > > > I could not get zero-copy to work with the i40e driver as it would crash. I've > > > attached the corresponding traces from dmesg. > > > > Thanks Ryan! I had a look at the crash, and it's in the XDP setup: > > > > i40e_xdp_setup: > > ... > > for (i = 0; i < vsi->num_queue_pairs; i++) > > WRITE_ONCE(vsi->rx_rings[i]->xdp_prog, vsi->xdp_prog); > > > > and the vsi->rx_ring[0] is NULL. This is clearly broken. > > > > It would help with more lines from your dmesg: the cut i40e log hints > > that something is really broken: > > > > [ 328.579154] i40e 0000:b7:00.2: failed to get tracking for 256 > > queues for VSI 0 err -12 > > [ 328.579280] i40e 0000:b7:00.2: setup of MAIN VSI failed > > [ 328.579367] i40e 0000:b7:00.2: can't remove VEB 162 with 0 VSIs left > > > > Is it possible to dig out the complete log? > > Hi Björn, > > I've linked a full dmesg log from an XDP setup crash. Note that there are > two i40e cards on this machine. The X710 (0000:65:00.0, 0000:65:00.1) works > fine, the X722 (0000:b7:00.0, 0000:b7:00.1, 0000:b7:00.2, 0000:b7:00.3) is the > one that is crashing on XDP setup. Ryan, I was wondering if you could run two small experiments since I cannot reproduce this? 1: Run your program using the two ports on your X710 card. Does it work? This is my setup and works for me. 2: On your Mellanox setup, insert a kick_tx() call for each of your two sockets before the poll() call in your forward() function. Just to see if it works when we explicitly wake up the driver. Thanks: Magnus > https://gitlab.com/mergetb/tech/network-emulation/kernel/snippets/1931080 > > Some info that may be useful: > > ry@turbine:~$ sudo ethtool -i eno7 > driver: i40e > version: 2.8.20-k > firmware-version: 3.33 0x80001006 1.1747.0 > expansion-rom-version: > bus-info: 0000:b7:00.2 > supports-statistics: yes > supports-test: yes > supports-eeprom-access: yes > supports-register-dump: yes > supports-priv-flags: yes > > The firmware version 3.33 was the latest I could find as of a few weeks ago. > > ry@turbine:~$ sudo lspci -vvv | grep 722 > b7:00.0 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GBASE-T (rev 04) > DeviceName: Intel LAN X722 #1 > Subsystem: Super Micro Computer Inc Ethernet Connection X722 for 10GBASE-T > b7:00.1 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GBASE-T (rev 04) > DeviceName: Intel LAN X722 #2 > Subsystem: Super Micro Computer Inc Ethernet Connection X722 for 10GBASE-T > b7:00.2 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ (rev 04) > DeviceName: Intel LAN X722 #3 > Subsystem: Super Micro Computer Inc Ethernet Connection X722 for 10GbE SFP+ > b7:00.3 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ (rev 04) > DeviceName: Intel LAN X722 #4 > Subsystem: Super Micro Computer Inc Ethernet Connection X722 for 10GbE SFP+ > > ry@ryzen:~$ uname -a > Linux ryzen 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11) x86_64 GNU/Linux > > ry@turbine:~/kmoa/bpf-next$ git log -2 > commit 60d71397d27e7859fdaaaaab6594e4d977ae46e2 (HEAD -> master) > Author: Ryan Goodfellow <rgoodfel@xxxxxxx> > Date: Wed Jan 15 16:54:39 2020 -0500 > > add xdpsock_multidev sample program > > This is a simple program that uses AF_XDP sockets to forward packets > between two interfaces using a common memory region and no copying of > packets. > > Signed-off-by: Ryan Goodfellow <rgoodfel@xxxxxxx> > > commit 9173cac3b64e6785dd604f5075e6035b045a0026 (origin/master, origin/HEAD) > Author: Andrii Nakryiko <andriin@xxxxxx> > Date: Wed Jan 15 11:08:56 2020 -0800 > > libbpf: Support .text sub-calls relocations > > The LLVM patch https://reviews.llvm.org/D72197 makes LLVM emit function call > relocations within the same section. This includes a default .text section, > which contains any BPF sub-programs. This wasn't the case before and so libbpf > was able to get a way with slightly simpler handling of subprogram call > relocations. > > This patch adds support for .text section relocations. It needs to ensure > correct order of relocations, so does two passes: > - first, relocate .text instructions, if there are any relocations in it; > - then process all the other programs and copy over patched .text instructions > for all sub-program calls. > > v1->v2: > - break early once .text program is processed. > > Signed-off-by: Andrii Nakryiko <andriin@xxxxxx> > Signed-off-by: Alexei Starovoitov <ast@xxxxxxxxxx> > Acked-by: Yonghong Song <yhs@xxxxxx> > Cc: Alexei Starovoitov <ast@xxxxxxxxxx> > Link: https://lore.kernel.org/bpf/20200115190856.2391325-1-andriin@xxxxxx > > -- > ~ ry
|