On Wed, May 26, 2021 at 3:59 AM Jesper Dangaard Brouer <brouer@xxxxxxxxxx> wrote: > > Hi All, > > I see a need for a driver to use different XDP metadata layout on a per > packet basis. E.g. PTP packets contains a hardware timestamp. E.g. VLAN > offloading and associated metadata as only relevant for packets using > VLANs. (Reserving room for every possible HW-hint is against the idea > of BTF). > > The question is how to support multiple BTF types on per packet basis? > (I need input from BTF experts, to tell me if I'm going in the wrong > direction with below ideas). I'm trying to follow all three threads, but still, can someone dumb it down for me and use few very specific examples to show how all this is supposed to work end-to-end. I.e., how the C definition for those custom BTF layouts might look like and how they are used in BPF programs, etc. I'm struggling to put all the pieces together, even ignoring all the netdev-specific configuration questions. As for BTF on a per-packet basis. This means that BTF itself is not known at the BPF program verification time, so there will be some sort of if/else if/else conditions to handle all recognized BTF IDs? Is that right? Fake but specific code would help (at least me) to actually join the discussion. Thanks. > > Let me describe a possible/proposed packet flow (feel free to disagree): > > When driver RX e.g. a PTP packet it knows HW is configured for PTP-TS and > when it sees a TS is available, then it chooses a code path that use the > BTF layout that contains RX-TS. To communicate what BTF-type the > XDP-metadata contains, it simply store the BTF-ID in xdp_buff->btf_id. > > When redirecting the xdp_buff is converted to xdp_frame, and also contains > the btf_id member. When converting xdp_frame to SKB, then netcore-code > checks if this BTF-ID have been registered, if so there is a (callback or > BPF-hook) registered to handle this BTF-type that transfer the fields from > XDP-metadata area into SKB fields. > > The XDP-prog also have access to this ctx->btf_id and can multiplex on > this in the BPF-code itself. Or use other methods like parsing PTP packet > and extract TS as expected BTF offset in XDP metadata (perhaps add a > sanity check if metadata-size match). > > > I talked to AF_XDP people (Magnus, Bjørn and William) about this idea, > and they pointed out that AF_XDP also need to know what BTF-layout is > used. As Magnus wrote in other thread; there is only 32-bit left in > AF_XDP descriptor option. We could store the BTF-ID in this field, but > it would block for other use-cases. Bjørn came up with the idea of > storing the BTF-ID in the BTF-layout itself, but as the last-member (to > have fixed offset to check in userspace AF_XDP program). Then we only > need to use a single bit in AF_XDP descriptor option to say > XDP-metadata is BTF described. > > In the AF_XDP userspace program, the programmers can have a similar > callback system per known BTF-ID. This way they can compile efficient > code per ID via requesting the BTF layout from the kernel. (Hint: > `bpftool btf dump id 42 format c`). > > Please let me know if this it the right or wrong direction? > > -- > Best regards, > Jesper Dangaard Brouer > MSc.CS, Principal Kernel Engineer at Red Hat > LinkedIn: http://www.linkedin.com/in/brouer >
