On 18/03/22 00:26, Jay Vosburgh wrote:
Jesse Brandeburg <jesse.brandeburg@xxxxxxxxx> wrote:On 3/17/2022 8:34 AM, Federico Parola wrote:We observed similar "batching" behavior on i40e devices late last year in ordinary use (not XDP, but using SR-IOV VFs). We instrumented the drivers at the send and receive sides, and determined that it appeared to be a behavior of the receiving device itself, i.e., packets 1 - 7 would be held indefinitely (as I recall, no interrupt or update of the RX ring pointers) until packet 8 arrives, at which point all 8 were delivered simultaneously. The issue was evidently in the firmware, and was resolved after a firmware upgrade.Hi Jay, I just updated the firmware to the latest version (v8.50 from v8.30) but unfortunately the problem is still there. However I'm experiencing the problem only when using AF_XDP in busy poll mode, all other modes (standard AF_XDP and normal packet reception) work just fine. Maybe the two problems are correlated in some way.This sounds related to the WB_ON_ITR feature in our hardware. If interrupts are disabled the driver needs to set that bit (and an ITR value) so that packets get written back in a timely manner and don't just wait for a cache line edge (I bet your Cache Line Size value in PCIe space (lspci) is set to 128?)FWIW, in the case we had, driver changes (5.14 upstream, the then-current Intel out of tree driver) didn't change the behavior. The PCI Cache Line Size was 64 bytes; the device was 13:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 02) Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 562SFP+ Adapter From an lspci -vvv perspective, failing and non-failing cards differed only in the serial numbers.
I tested it on 3 different XL710 NICs, all updated to the latest 8.50 firmware, and they all present the same 8 packets "batching" behavior. How do I check the PCI Cache Line Size (I didn't find such entry in lspci -vvv)? Normal cache line size is 64B on my system, but I guess they are two different things.
Maybe something to chase there? We may also be able to look into it if it wasn't fixed already upstream.-J --- -Jay Vosburgh, jay.vosburgh@xxxxxxxxxxxxx
|