Magnus Karlsson <magnus.karlsson@xxxxxxxxx> writes: > On Wed, 12 Apr 2023 at 15:40, Toke Høiland-Jørgensen <toke@xxxxxxxxxx> wrote: >> >> Kal Cutter Conley <kal.conley@xxxxxxxxxxx> writes: >> >> >> > > Add core AF_XDP support for chunk sizes larger than PAGE_SIZE. This >> >> > > enables sending/receiving jumbo ethernet frames up to the theoretical >> >> > > maxiumum of 64 KiB. For chunk sizes > PAGE_SIZE, the UMEM is required >> >> > > to consist of HugeTLB VMAs (and be hugepage aligned). Initially, only >> >> > > SKB mode is usable pending future driver work. >> >> > >> >> > Hmm, interesting. So how does this interact with XDP multibuf? >> >> >> >> To me it currently does not interact with mbuf in any way as it is enabled >> >> only for skb mode which linearizes the skb from what i see. >> >> >> >> I'd like to hear more about Kal's use case - Kal do you use AF_XDP in SKB >> >> mode on your side? >> > >> > Our use-case is to receive jumbo Ethernet frames up to 9000 bytes with >> > AF_XDP in zero-copy mode. This patchset is a step in this direction. >> > At the very least, it lets you test out the feature in SKB mode >> > pending future driver support. Currently, XDP multi-buffer does not >> > support AF_XDP at all. It could support it in theory, but I think it >> > would need some UAPI design work and a bit of implementation work. >> > >> > Also, I think that the approach taken in this patchset has some >> > advantages over XDP multi-buffer: >> > (1) It should be possible to achieve higher performance >> > (a) because the packet data is kept together >> > (b) because you need to acquire and validate less descriptors >> > and touch the queue pointers less often. >> > (2) It is a nicer user-space API. >> > (a) Since the packet data is all available in one linear >> > buffer. This may even be a requirement to avoid an extra copy if the >> > data must be handed off contiguously to other code. >> > >> > The disadvantage of this patchset is requiring the user to allocate >> > HugeTLB pages which is an extra complication. >> > >> > I am not sure if this patchset would need to interact with XDP >> > multi-buffer at all directly. Does anyone have anything to add here? >> >> Well, I'm mostly concerned with having two different operation and >> configuration modes for the same thing. We'll probably need to support >> multibuf for AF_XDP anyway for the non-ZC path, which means we'll need >> to create a UAPI for that in any case. And having two APIs is just going >> to be more complexity to handle at both the documentation and >> maintenance level. > > One does not replace the other. We need them both, unfortunately. > Multi-buff is great for e.g., stitching together different headers > with the same data. Point to different buffers for the header in each > packet but the same piece of data in all of them. This will never be > solved with Kal's approach. We just need multi-buffer support for > this. BTW, we are close to posting multi-buff support for AF_XDP. Just > hang in there a little while longer while the last glitches are fixed. > We have to stage it in two patch sets as it will be too long > otherwise. First one will only contain improvements to the xsk > selftests framework so that multi-buffer tests can be supported. The > second one will be the core code and the actual multi-buffer tests. Alright, sounds good! > As for what Kal's patches are good for, please see below. > >> It *might* be worth it to do this if the performance benefit is really >> compelling, but, well, you'd need to implement both and compare directly >> to know that for sure :) > > The performance benefit is compelling. As I wrote in a mail to a post > by Kal, there are users out there that state that this feature (for > zero-copy mode nota bene) is a must for them to be able to use AF_XDP > instead of DPDK style user-mode drivers. They have really tough > latency requirements. Hmm, okay, looking forward to seeing the benchmark results, then! :) -Toke
