On Mon, Feb 18, 2019 at 10:38 AM Daniel Borkmann <daniel@xxxxxxxxxxxxx> wrote: > > On 02/18/2019 09:20 AM, Magnus Karlsson wrote: > > On Fri, Feb 15, 2019 at 5:48 PM Daniel Borkmann <daniel@xxxxxxxxxxxxx> wrote: > >> > >> On 02/13/2019 12:55 PM, Jesper Dangaard Brouer wrote: > >>> On Wed, 13 Feb 2019 12:32:47 +0100 > >>> Magnus Karlsson <magnus.karlsson@xxxxxxxxx> wrote: > >>>> On Mon, Feb 11, 2019 at 9:44 PM Jonathan Lemon <jonathan.lemon@xxxxxxxxx> wrote: > >>>>> On 8 Feb 2019, at 5:05, Magnus Karlsson wrote: > >>>>> > >>>>>> This patch proposes to add AF_XDP support to libbpf. The main reason > >>>>>> for this is to facilitate writing applications that use AF_XDP by > >>>>>> offering higher-level APIs that hide many of the details of the AF_XDP > >>>>>> uapi. This is in the same vein as libbpf facilitates XDP adoption by > >>>>>> offering easy-to-use higher level interfaces of XDP > >>>>>> functionality. Hopefully this will facilitate adoption of AF_XDP, make > >>>>>> applications using it simpler and smaller, and finally also make it > >>>>>> possible for applications to benefit from optimizations in the AF_XDP > >>>>>> user space access code. Previously, people just copied and pasted the > >>>>>> code from the sample application into their application, which is not > >>>>>> desirable. > >>>>> > >>>>> I like the idea of encapsulating the boilerplate logic in a library. > >>>>> > >>>>> I do think there is an important missing piece though - there should be > >>>>> some code which queries the netdev for how many queues are attached, and > >>>>> create the appropriate number of umem/AF_XDP sockets. > >>>>> > >>>>> I ran into this issue when testing the current AF_XDP code - on my test > >>>>> boxes, the mlx5 card has 55 channels (aka queues), so when the test program > >>>>> binds only to channel 0, nothing works as expected, since not all traffic > >>>>> is being intercepted. While obvious in hindsight, this took a while to > >>>>> track down. > >>>> > >>>> Yes, agreed. You are not the first one to stumble upon this problem > >>>> :-). Let me think a little bit on how to solve this in a good way. We > >>>> need this to be simple and intuitive, as you say. > >>> > >>> I see people hitting this with AF_XDP all the time... I had some > >>> backup-slides[2] in our FOSDEM presentation[1] that describe the issue, > >>> give the performance reason why and propose a workaround. > >> > >> Magnus, I presume you're going to address this for the initial libbpf merge > >> since the plan is to make it easier to consume for users? > > > > I think the first thing we need is education and documentation. Have a > > FAQ or "common mistakes" section in the Documentation. And of course, > > sending Jesper around the world reminding people about this ;-). > > > > To address this on a libbpf interface level, I think the best way is > > to reprogram the NIC to send all traffic to the queue that you > > provided in the xsk_socket__create call. This "set up NIC routing" > > behavior can then be disable with a flag, just as the XDP program > > loading can be disabled. The standard config of xsk_socket__create > > will then set up as many things for the user as possible just to get > > up and running quickly. More advanced users can then disable parts of > > it to gain more flexibility. Does this sound OK? Do not want to go the > > route of polling multiple sockets and aggregating the traffic as this > > will have significant negative performance implications. > > I think that is fine, I would probably make this one a dedicated API call > in order to have some more flexibility than just simple flag. E.g. once > nfp AF_XDP support lands at some point, I could imagine that this call > resp. a drop-in replacement API call for more advanced steering could > also take an offloaded BPF prog fd, for example, which then would program > the steering on the NIC [0]. Seems at least there's enough complexity on > its own to have a dedicated API for it. Thoughts? I agree that there is probably enough complexity to warrant adding a higher level API to deal with this problem (flow steering). But there are likely a number of cases we have not thought that would complicate it even further. This is why I suggest that this functionality should be in its own patch set that I can devote some time and thought to. IMO, the current patch set and functionality does already lower the bar of entry significantly and has a value even without hiding or controlling the steering of traffic. What I would like to do in this patch set is to add a FAQ section in Documentation/networking/af_xdp.rst explaining this problem. Something like: "Q: Why am I not seeing any traffic? A: Check these four things.....". Could add some text in the libbpf README referring to this document also. Opinions? Thanks: Magnus > Thanks, > Daniel > > [0] https://patchwork.ozlabs.org/cover/910614/
|