On Fri, Oct 6, 2023 at 5:23 PM Magnus Karlsson <magnus.karlsson@xxxxxxxxx> wrote: > > On Fri, 6 Oct 2023 at 13:41, Srivats P <pstavirs@xxxxxxxxx> wrote: > > > > On Thu, Oct 5, 2023 at 9:03 PM Magnus Karlsson > > <magnus.karlsson@xxxxxxxxx> wrote: > > > > > > On Thu, 5 Oct 2023 at 17:18, Srivats P <pstavirs@xxxxxxxxx> wrote: > > > > > > > > On Thu, Oct 5, 2023 at 8:40 PM Magnus Karlsson > > > > <magnus.karlsson@xxxxxxxxx> wrote: > > > > > > > > > > On Thu, 5 Oct 2023 at 16:31, Srivats P <pstavirs@xxxxxxxxx> wrote: > > > > > > > > > > > > Hi, > > > > > > > > > > > > I want to clarify my AF_XDP understanding for a particular scenario. > > > > > > > > > > > > Consider a single UMEM with 1 Fill Ring and 1 Completion Ring and > > > > > > single XDP socket bound to queue 0 with 1 Tx Ring but NO Rx Ring. > > > > > > Assume the NIC has only a single queue (to keep things simple for > > > > > > explaining this scenario). > > > > > > > > > > > > There is a XDP program attached to Queue 0 which does either a > > > > > > XDP_DROP or XDP_PASS for all Rx packets. > > > > > > > > > > > > We are running in Driver mode. > > > > > > > > > > > > What happens to RX packets received on Queue 0? > > > > > > > > > > > > Here's my understanding - > > > > > > > > > > > > * Even though there is no AF_XDP Rx Ring, there will be a NIC Rx Ring > > > > > > for queue 0 > > > > > > * The NIC Rx Ring for queue 0 is populated by the driver with UMEM > > > > > > buffers taken from the Fill ring > > > > > > > > > > In driver mode (i.e. not zero-copy mode) this will not happen. The > > > > > ring will be populated by kernel buffers. > > > > > > > > What is the behaviour in zero-copy mode? > > > > > > Then the behaviour is according to what you wrote above. Just note > > > that nothing will be returned in the fill ring as you are not sending > > > anything to the AF_XDP socket. > > > > I assume you meant "Rx Ring" and not "Fill ring" above? > > Yes, sorry. Nothing will arrive in the Rx ring, and the Fill ring will > not be used apart from the initial burst of buffers that will be > grabbed by the system. Note though that you have to provide some > buffers in the fill ring as these are the ones that are used by the > NIC in zero-copy mode. They are just never returned to user-space > through the Rx ring. They are just recycled in the driver so the > system never needs to grab more in your scenario. > After a XDP_DROP or XDP_PASS at RX, will the driver recycle the umem buffer back into the NIC's RX-ring or will it put it on the CQ? In case of the latter, the FQ will eventually run out of buffers. Is this behaviour driver specific (recycle to NIC RX-Ring or CQ) or same for all drivers in ZC mode? > > > > > > > > > > > > > > * A Rx packet will be received in the NIC Rx Ring for Queue 0 first > > > > > > * The driver will run the XDP program on the Rx packet buffer (a UMEM buffer) > > > > > > * If the program results in XDP_DROP, the driver will "free" the Umem > > > > > > buffer by putting it on the Completion Ring > > > > > > * If the program results in XDP_PASS, the driver will allocate a > > > > > > standard Linux kernel SKB, copy the packet buffer contents into the > > > > > > SKB and queue it up for standard netdev processing; It will then > > > > > > "free" the RX Umem buffer by putting it on the Completion Ring (since > > > > > > we have already copied packet into the skb) > > > > > > > > > > As the buffers are kernel buffers, user-space will not be notified. > > > > > The completion ring is solely for the Tx path, saying that user space > > > > > can have the buffer back. > > > > > > > > > > The rest is correct. > > > > > > > > > > /Magnus > > > > > > > > > > > Is this understanding correct or am I mistaken anywhere? > > > > > > > > > > > > Thanks in advance, > > > > > > Srivats
|