Hi,
Unfortunately, similar scalability issues that were addressed for XDP
processing in ice, exist for XDP in the zero-copy driver used by AF_XDP.
Let's resolve them in mostly the same way as we did in [0] and utilize
the Tx batching API from xsk buffer pool.
Move the array of Tx descriptors that is used with batching approach to
the xsk buffer pool. This means that future users of this API will not
have to carry the array on their own side, they can simply refer to
pool's tx_desc array.
Improve also the Rx side where we extend ice_alloc_rx_buf_zc() to handle
the ring wrap and bump Rx tail more frequently. By doing so, Rx side is
adjusted to Tx and it was needed for l2fwd scenario.
Here are the improvements of performance numbers that this set brings
measured with xdpsock app in busy poll mode for 1 and 2 core modes.
Both Tx and Rx rings were sized to 1k length and busy poll budget was
256.
----------------------------------------------------------------
| txonly: | l2fwd | rxdrop
----------------------------------------------------------------
1C | 149% | 14% | 3%
----------------------------------------------------------------
2C | 134% | 20% | 5%
----------------------------------------------------------------
Next step will be to introduce batching onto Rx side.
v4 - address Alexandr's review:
* new patch (2) for making sure ring size is pow(2) when attaching
xsk socket
* don't open code ALIGN_DOWN (patch 3)
* resign from storing tx_thresh in ice_tx_ring (patch 4)
* scope variables in a better way for Tx batching (patch 7)
v3:
* drop likely() that was wrapping napi_complete_done (patch 1)
* introduce configurable Tx threshold (patch 2)
* handle ring wrap on Rx side when allocating buffers (patch 3)
* respect NAPI budget when cleaning Tx descriptors in ZC (patch 6)
v2:
* introduce new patch that resets @next_dd and @next_rs fields
* use batching API for AF_XDP Tx on ice side
Thanks,
Maciej
[0]: https://lore.kernel.org/bpf/20211015162908.145341-8-anthony.l.nguyen@xxxxxxxxx/
Maciej Fijalkowski (7):
ice: remove likely for napi_complete_done
ice: xsk: force rings to be sized to power of 2
ice: xsk: handle SW XDP ring wrap and bump tail more often
ice: make Tx threshold dependent on ring length
ice: xsk: avoid potential dead AF_XDP Tx processing
ice: xsk: improve AF_XDP ZC Tx and use batching API
ice: xsk: borrow xdp_tx_active logic from i40e
Magnus Karlsson (1):
i40e: xsk: move tmp desc array from driver to pool
drivers/net/ethernet/intel/i40e/i40e_txrx.c | 11 -
drivers/net/ethernet/intel/i40e/i40e_txrx.h | 1 -
drivers/net/ethernet/intel/i40e/i40e_xsk.c | 4 +-
drivers/net/ethernet/intel/ice/ice_ethtool.c | 2 +
drivers/net/ethernet/intel/ice/ice_main.c | 4 +-
drivers/net/ethernet/intel/ice/ice_txrx.c | 6 +-
drivers/net/ethernet/intel/ice/ice_txrx.h | 6 +-
drivers/net/ethernet/intel/ice/ice_txrx_lib.c | 15 +-
drivers/net/ethernet/intel/ice/ice_xsk.c | 375 +++++++++++++-----
drivers/net/ethernet/intel/ice/ice_xsk.h | 27 +-
include/net/xdp_sock_drv.h | 5 +-
include/net/xsk_buff_pool.h | 1 +
net/xdp/xsk.c | 13 +-
net/xdp/xsk_buff_pool.c | 7 +
net/xdp/xsk_queue.h | 12 +-
15 files changed, 329 insertions(+), 160 deletions(-)
--
2.33.1
