Hi, On Fri, Jun 24, 2022 at 06:51:33PM +0530, Srivats P wrote: > Hi, > > Using the xdpsock txonly benchmark in zero-copy mode with the i40e > driver, I get ~22 Mpps for 64 Byte packets. > > However, for 1518 byte packets, although the theoretical pps rate for > 40Gbps is ~3.25 Mpps (which is much lower than 22 Mpps), I am not able > to get the full rate - only around 39G or 39.5Gbps (i.e. < 3.25Mpps). > My assumption is AF_XDP performance is more pps dependent than packet > size related and so I thought full 40G should be achievable. > > Settings: > * tuned set to "throughput performance" > * hardware tx ring size set to 512 descriptors - increasing it to up > to the max of 4096 doesn't have any noticeable changes > > Increasing the packet size to 3000 or 4096 does bump up the Gbps rate > to around 39.9Gbps. > > Is there a way to get the full 40Gbps (or close, say 39.9Gbps) tx rate > with 1518 byte packets? While I don't know for the XDP part itself, you should really not speak in terms of bps but in terms of pps exclusively. The reason is that it is very difficult otherwise to know what layer you're measuring. In addition, 40 Gbps is 4*10GbE which itself is exactly 10G on copper but can be a bit more (10.3125G) on fiber depending on the type (and maybe a bit less on single-mode as well though I don't remember having seen that in a while), so taking that into account can make it even harder to observe an exactly round value. So assuming that you're running on copper with 40.000 Gbps, this is the on-wire bit rate, that includes the 8-byte preamble, the visible eth frame, the FCS (4 bytes) and the IFG (12 bytes), hence 1538 bytes for the regular 1514 bytes. This gives a maximum frame rate of 3250975 pps for such frames, which is what you'd get if by 1518 you included the FCS, or 3242542 pps if by 1518 you included a VLAN header (hence 1542 bytes on the wire). If we take that last example that sounds like the most plausible, that gives you a layer2 max bit rate of 3242542*1518*8 = 39.377 Gbps. As such, depending on how and where you measure, it's very possible that you've totally filled the wire. Thus as a conclusion: forget about bit rate and only count packet rates. Hoping this helps, Willy
|