From: Vladimir Oltean <vladimir.oltean@xxxxxxx>
This series adds support to the enetc driver for the basic XDP primitives.
The ENETC is a network controller found inside the NXP LS1028A SoC,
which is a dual-core Cortex A72 device for industrial networking,
with the CPUs clocked at up to 1.3 GHz. On this platform, there are 4
ENETC ports and a 6-port embedded DSA switch, in a topology that looks
like this:
+-------------------------------------------------------------------------+
| +--------+ 1 Gbps (typically disabled) |
| ENETC PCI | ENETC |--------------------------+ |
| Root Complex | port 3 |-----------------------+ | |
| Integrated +--------+ | | |
| Endpoint | | |
| +--------+ 2.5 Gbps | | |
| | ENETC |--------------+ | | |
| | port 2 |-----------+ | | | |
| +--------+ | | | | |
| | | | | |
| +------------------------------------------------+
| | | Felix | | Felix | |
| | Switch | port 4 | | port 5 | |
| | +--------+ +--------+ |
| | |
| +--------+ +--------+ | +--------+ +--------+ +--------+ +--------+ |
| | ENETC | | ENETC | | | Felix | | Felix | | Felix | | Felix | |
| | port 0 | | port 1 | | | port 0 | | port 1 | | port 2 | | port 3 | |
+-------------------------------------------------------------------------+
| | | | | |
v v v v v v
Up to Up to Up to 4x 2.5Gbps
2.5Gbps 1Gbps
The ENETC ports 2 and 3 can act as DSA masters for the embedded switch.
Because 4 out of the 6 externally-facing ports of the SoC are switch
ports, the most interesting use case for XDP on this device is in fact
XDP_TX on the 2.5Gbps DSA master.
Nonetheless, the results presented below are for IPv4 forwarding between
ENETC port 0 (eno0) and port 1 (eno1) both configured for 1Gbps.
There are two streams of IPv4/UDP datagrams with a frame length of 64
octets delivered at 100% port load to eno0 and to eno1. eno0 has a flow
steering rule to process the traffic on RX ring 0 (CPU 0), and eno1 has
a flow steering rule towards RX ring 1 (CPU 1).
For the IPFWD test, standard IP routing was enabled in the netns.
For the XDP_DROP test, the samples/bpf/xdp1 program was attached to both
eno0 and to eno1.
For the XDP_TX test, the samples/bpf/xdp2 program was attached to both
eno0 and to eno1.
For the XDP_REDIRECT test, the samples/bpf/xdp_redirect program was
attached once to the input of eno0/output of eno1, and twice to the
input of eno1/output of eno0.
Finally, the preliminary results are as follows:
| IPFWD | XDP_TX | XDP_REDIRECT | XDP_DROP
--------+-------+--------+-------------------------
fps | 761 | 2535 | 1735 | 2783
Gbps | 0.51 | 1.71 | 1.17 | n/a
There is a strange phenomenon in my testing sistem where it appears that
one CPU is processing more than the other. I have not investigated this
too much. Also, the code might not be very well optimized (for example
dma_sync_for_device is called with the full ENETC_RXB_DMA_SIZE_XDP).
Design wise, the ENETC is a PCI device with BD rings, so it uses the
MEM_TYPE_PAGE_SHARED memory model, as can typically be seen in Intel
devices. The strategy was to build upon the existing model that the
driver uses, and not change it too much. So you will see things like a
separate NAPI poll function for XDP.
I have only tested with PAGE_SIZE=4096, and since we split pages in
half, it means that MTU-sized frames are scatter/gather (the XDP
headroom + skb_shared_info only leaves us 1476 bytes of data per
buffer). This is sub-optimal, but I would rather keep it this way and
help speed up Lorenzo's series for S/G support through testing, rather
than change the enetc driver to use some other memory model like page_pool.
My code is already structured for S/G, and that works fine for XDP_DROP
and XDP_TX, just not for XDP_REDIRECT, even between two enetc ports.
So the S/G XDP_REDIRECT is stubbed out (the frames are dropped), but
obviously I would like to remove that limitation soon.
Please note that I am rather new to this kind of stuff, I am more of a
control path person, so I would appreciate feedback.
Enough talking, on to the patches.
Vladimir Oltean (9):
net: enetc: consume the error RX buffer descriptors in a dedicated
function
net: enetc: move skb creation into enetc_build_skb
net: enetc: add a dedicated is_eof bit in the TX software BD
net: enetc: clean the TX software BD on the TX confirmation path
net: enetc: move up enetc_reuse_page and enetc_page_reusable
net: enetc: add support for XDP_DROP and XDP_PASS
net: enetc: add support for XDP_TX
net: enetc: increase RX ring default size
net: enetc: add support for XDP_REDIRECT
drivers/net/ethernet/freescale/enetc/enetc.c | 826 +++++++++++++++---
drivers/net/ethernet/freescale/enetc/enetc.h | 53 +-
.../ethernet/freescale/enetc/enetc_ethtool.c | 19 +-
.../net/ethernet/freescale/enetc/enetc_pf.c | 2 +
4 files changed, 796 insertions(+), 104 deletions(-)
--
2.25.1
