Federico Parola <federico.parola@xxxxxxxxx> wrote: >Hello everybody, >I'm experiencing a strange problem when running an AF_XDP application with >busy poll enabled on a Intel XL710 NIC (i40e driver). >The problem can be replicated running the xdpsock kernel sample in rx or >l2fwd mode. >The first packet I send to the machine is correctly received by the >application. After this, packets are only received in batches of 8. >If I send 7 packets the application sees nothing, while the 8th one >triggers the reception of all 8 packets. >Disabling the busy poll mode everything works fine and packets are >immediately received as they are sent. > >I tried changing kernel (5.12, 5.14 and 5.16) but all present the same >problem. >I also tried using another NIC, an Intel X540 with ixgbe driver and the >problem isn't there, so I guess is NIC/driver related. > >I tried monitoring ethtool statistics. When sending packets between 1 and >7 these counters are increased: >stat: 64 ( 64) <= port.rx_bytes /sec >stat: 1 ( 1) <= port.rx_size_64 /sec >stat: 1 ( 1) <= port.rx_unicast /sec >stat: 1 ( 1) <= rx_unicast /sec > >While the 8th one triggers this updates: >stat: 64 ( 64) <= port.rx_bytes /sec >stat: 1 ( 1) <= port.rx_size_64 /sec >stat: 1 ( 1) <= port.rx_unicast /sec >stat: 477 ( 477) <= rx-0.bytes /sec >stat: 8 ( 8) <= rx-0.packets /sec >stat: 477 ( 477) <= rx_bytes /sec >stat: 8 ( 8) <= rx_packets /sec >stat: 1 ( 1) <= rx_unicast /sec > >As far as I understand the first set of counters are hardware counters, so >it makes me think that packets are kept in the NIC and not even sent to >memory. > >Does anyone have any suggestion on what could be causing this problem? >Does enabling busy poll set some flag on the NIC? We observed similar "batching" behavior on i40e devices late last year in ordinary use (not XDP, but using SR-IOV VFs). We instrumented the drivers at the send and receive sides, and determined that it appeared to be a behavior of the receiving device itself, i.e., packets 1 - 7 would be held indefinitely (as I recall, no interrupt or update of the RX ring pointers) until packet 8 arrives, at which point all 8 were delivered simultaneously. The issue was evidently in the firmware, and was resolved after a firmware upgrade. -J > >Best regards, >Federico Parola --- -Jay Vosburgh, jay.vosburgh@xxxxxxxxxxxxx
|