Re: XDP multi-buffer incl. jumbo-frames (Was: [RFC V1 net-next 1/1] net: ena: implement XDP drop support)

["Eelco Chaudron" <echaudro@xxxxxxxxxx>]

On 28 Jun 2019, at 9:46, Toke Høiland-Jørgensen wrote:

> "Eelco Chaudron" <echaudro@xxxxxxxxxx> writes:
>
> > On 26 Jun 2019, at 10:38, Jesper Dangaard Brouer wrote:
> >
> > > On Tue, 25 Jun 2019 03:19:22 +0000
> > > "Machulsky, Zorik" <zorik@xxxxxxxxxx> wrote:
> > >
> > > > On 6/23/19, 7:21 AM, "Jesper Dangaard Brouer" 
> > > > <brouer@xxxxxxxxxx>
> > > > wrote:
> > > >
> > > >     On Sun, 23 Jun 2019 10:06:49 +0300 <sameehj@xxxxxxxxxx> wrote:
> > > >
> > > >     > This commit implements the basic functionality of drop/pass
> > > > logic in the
> > > >     > ena driver.
> > > >
> > > >     Usually we require a driver to implement all the XDP return
> > > > codes,
> > > >     before we accept it.  But as Daniel and I discussed with Zorik
> > > > during
> > > >     NetConf[1], we are going to make an exception and accept the
> > > > driver
> > > >     if you also implement XDP_TX.
> > > >
> > > >     As we trust that Zorik/Amazon will follow and implement
> > > > XDP_REDIRECT
> > > >     later, given he/you wants AF_XDP support which requires
> > > > XDP_REDIRECT.
> > > >
> > > > Jesper, thanks for your comments and very helpful discussion during
> > > > NetConf! That's the plan, as we agreed. From our side I would like 
> > > > to
> > > > reiterate again the importance of multi-buffer support by xdp 
> > > > frame.
> > > > We would really prefer not to see our MTU shrinking because of xdp
> > > > support.
> > >
> > > Okay we really need to make a serious attempt to find a way to 
> > > support
> > > multi-buffer packets with XDP. With the important criteria of not
> > > hurting performance of the single-buffer per packet design.
> > >
> > > I've created a design document[2], that I will update based on our
> > > discussions: [2]
> > > https://github.com/xdp-project/xdp-project/blob/master/areas/core/xdp-multi-buffer01-design.org
> > >
> > > The use-case that really convinced me was Eric's packet 
> > > header-split.
> > >
> > >
> > > Lets refresh: Why XDP don't have multi-buffer support:
> > >
> > > XDP is designed for maximum performance, which is why certain
> > > driver-level
> > > use-cases were not supported, like multi-buffer packets (like
> > > jumbo-frames).
> > > As it e.g. complicated the driver RX-loop and memory model handling.
> > >
> > > The single buffer per packet design, is also tied into eBPF
> > > Direct-Access
> > > (DA) to packet data, which can only be allowed if the packet memory 
> > > is
> > > in
> > > contiguous memory.  This DA feature is essential for XDP 
> > > performance.
> > >
> > >
> > > One way forward is to define that XDP only get access to the first
> > > packet buffer, and it cannot see subsequent buffers.  For XDP_TX and
> > > XDP_REDIRECT to work then XDP still need to carry pointers (plus
> > > len+offset) to the other buffers, which is 16 bytes per extra 
> > > buffer.
> >
> >
> > I’ve seen various network processor HW designs, and they normally 
> > get
> > the first x bytes (128 - 512) which they can manipulate
> > (append/prepend/insert/modify/delete).
> >
> > There are designs where they can “page in” the additional 
> > fragments
> > but it’s expensive as it requires additional memory transfers. But 
> > the
> > majority do not care (cannot change) the remaining fragments. Can 
> > also
> > not think of a reason why you might want to remove something at the 
> > end
> > of the frame (thinking about routing/forwarding needs here).
> >
> > If we do want XDP to access other fragments we could do this through 
> > a
> > helper which swaps the packet context?
>
> Yeah, I was also going to suggest a helper for that. It doesn't
> necessarily need to swap the packet context, it could just return a 
> new
> pointer?

Yes that will work, my head was still thinking ASICs where there is 
limited SRAM space…