On 3/18/2022 11:33 AM, Jay Vosburgh wrote:
I tested it on 3 different XL710 NICs, all updated to the latest 8.50 firmware, and they all present the same 8 packets "batching" behavior. How do I check the PCI Cache Line Size (I didn't find such entry in lspci -vvv)? Normal cache line size is 64B on my system, but I guess they are two different things.It's in the lspci -vvv, near the top, e.g., 08:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) Subsystem: Intel Corporation Ethernet Converged Network Adapter XL710-Q2 Physical Slot: 1 Control: [...bunch of stuff...] Status: [...bunch of stuff...] Latency: 0, Cache Line Size: 64 bytes Checking the PCIe spec, though, I'm not sure why this would matter, as PCIe 4.0 7.5.1.3 Cache Line Size Register says it "has no effect on any PCI Express device behavior."
Thanks Jay, that might just be the sound of my brain short circuiting, as I thought that held some relevance for some reason. Our receive descriptors are 16 bytes (with options for 32 bytes sometimes), and the transmit are 16. 8*16=128, so I was suspecting something related to cache line size in the PCIe is triggering a writeback when we get to that boundary, it's probably some other trigger I don't remember.
I know the XL710 has a documented erratum where while interrupts are disabled (like while polling) that hardware won't write back until it hits a writeback criteria, or unless the interrupt is re-enabled. We have a bunch of workarounds in the driver trying to address this, I was thinking maybe one of those workarounds isn't hitting in the XDP code.
Oh, and WB_ON_ITR didn't appear until the X722 generation, so it's not there on XL710. Hope this helps!
Jesse
|