"Eelco Chaudron" <echaudro@xxxxxxxxxx> writes: > On 26 Jun 2019, at 10:38, Jesper Dangaard Brouer wrote: > >> On Tue, 25 Jun 2019 03:19:22 +0000 >> "Machulsky, Zorik" <zorik@xxxxxxxxxx> wrote: >> >>> On 6/23/19, 7:21 AM, "Jesper Dangaard Brouer" <brouer@xxxxxxxxxx> >>> wrote: >>> >>> On Sun, 23 Jun 2019 10:06:49 +0300 <sameehj@xxxxxxxxxx> wrote: >>> >>> > This commit implements the basic functionality of drop/pass >>> logic in the >>> > ena driver. >>> >>> Usually we require a driver to implement all the XDP return >>> codes, >>> before we accept it. But as Daniel and I discussed with Zorik >>> during >>> NetConf[1], we are going to make an exception and accept the >>> driver >>> if you also implement XDP_TX. >>> >>> As we trust that Zorik/Amazon will follow and implement >>> XDP_REDIRECT >>> later, given he/you wants AF_XDP support which requires >>> XDP_REDIRECT. >>> >>> Jesper, thanks for your comments and very helpful discussion during >>> NetConf! That's the plan, as we agreed. From our side I would like to >>> reiterate again the importance of multi-buffer support by xdp frame. >>> We would really prefer not to see our MTU shrinking because of xdp >>> support. >> >> Okay we really need to make a serious attempt to find a way to support >> multi-buffer packets with XDP. With the important criteria of not >> hurting performance of the single-buffer per packet design. >> >> I've created a design document[2], that I will update based on our >> discussions: [2] >> https://github.com/xdp-project/xdp-project/blob/master/areas/core/xdp-multi-buffer01-design.org >> >> The use-case that really convinced me was Eric's packet header-split. >> >> >> Lets refresh: Why XDP don't have multi-buffer support: >> >> XDP is designed for maximum performance, which is why certain >> driver-level >> use-cases were not supported, like multi-buffer packets (like >> jumbo-frames). >> As it e.g. complicated the driver RX-loop and memory model handling. >> >> The single buffer per packet design, is also tied into eBPF >> Direct-Access >> (DA) to packet data, which can only be allowed if the packet memory is >> in >> contiguous memory. This DA feature is essential for XDP performance. >> >> >> One way forward is to define that XDP only get access to the first >> packet buffer, and it cannot see subsequent buffers. For XDP_TX and >> XDP_REDIRECT to work then XDP still need to carry pointers (plus >> len+offset) to the other buffers, which is 16 bytes per extra buffer. > > > I’ve seen various network processor HW designs, and they normally get > the first x bytes (128 - 512) which they can manipulate > (append/prepend/insert/modify/delete). > > There are designs where they can “page in” the additional fragments > but it’s expensive as it requires additional memory transfers. But the > majority do not care (cannot change) the remaining fragments. Can also > not think of a reason why you might want to remove something at the end > of the frame (thinking about routing/forwarding needs here). > > If we do want XDP to access other fragments we could do this through a > helper which swaps the packet context? Yeah, I was also going to suggest a helper for that. It doesn't necessarily need to swap the packet context, it could just return a new pointer? -Toke