On Fri, Oct 4, 2019 at 3:47 PM Andrii Nakryiko <andrii.nakryiko@xxxxxxxxx> wrote: > > On Fri, Oct 4, 2019 at 2:58 PM Daniel Borkmann <daniel@xxxxxxxxxxxxx> wrote: > > > > On Fri, Oct 04, 2019 at 11:06:13PM +0200, Daniel Borkmann wrote: > > > On Fri, Oct 04, 2019 at 01:21:55PM -0700, Andrii Nakryiko wrote: > > > > On Fri, Oct 4, 2019 at 11:30 AM Jakub Kicinski > > > > <jakub.kicinski@xxxxxxxxxxxxx> wrote: > > > > > On Fri, 4 Oct 2019 09:00:42 -0700, Andrii Nakryiko wrote: > > > > > > On Fri, Oct 4, 2019 at 8:44 AM David Ahern <dsahern@xxxxxxxxx> wrote: > > > > > >> > I'm not following you; my interpretation of your comment seems like you > > > > > > > are making huge assumptions. > > > > > > > > > > > > > > I build bpf programs for specific kernel versions using the devel > > > > > > > packages for the specific kernel of interest. > > > > > > > > > > > > Sure, and you can keep doing that, just don't include bpf_helpers.h? > > > > > > > > > > > > What I was saying, though, especially having in mind tracing BPF > > > > > > programs that need to inspect kernel structures, is that it's quite > > > > > > impractical to have to build many different versions of BPF programs > > > > > > for each supported kernel version and distribute them in binary form. > > > > > > So people usually use BCC and do compilation on-the-fly using BCC's > > > > > > embedded Clang. > > > > > > > > > > > > BPF CO-RE is providing an alternative, which will allow to pre-compile > > > > > > your program once for many different kernels you might be running your > > > > > > program on. There is tooling that eliminates the need for system > > > > > > headers. Instead we pre-generate a single vmlinux.h header with all > > > > > > the types/enums/etc, that are then used w/ BPF CO-RE to build portable > > > > > > BPF programs capable of working on multiple kernel versions. > > > > > > > > > > > > So what I was pointing out there was that this vmlinux.h would be > > > > > > ideally generated from latest kernel and not having latest > > > > > > BPF_FUNC_xxx shouldn't be a problem. But see below about situation > > > > > > being worse. > > > > > > > > > > Surely for distroes tho - they would have kernel headers matching the > > > > > kernel release they ship. If parts of libbpf from GH only work with > > > > > the latest kernel, distroes should ship libbpf from the kernel source, > > > > > rather than GH. > > > > > > > > > > > > > Nevertheless, it is a problem and thanks for bringing it up! I'd say > > > > > > > > for now we should still go ahead with this move and try to solve with > > > > > > > > issue once bpf_helpers.h is in libbpf. If bpf_helpers.h doesn't work > > > > > > > > for someone, it's no worse than it is today when users don't have > > > > > > > > bpf_helpers.h at all. > > > > > > > > > > > > > > > > > > > > > > If this syncs to the github libbpf, it will be worse than today in the > > > > > > > sense of compile failures if someone's header file ordering picks > > > > > > > libbpf's bpf_helpers.h over whatever they are using today. > > > > > > > > > > > > Today bpf_helpers.h don't exist for users or am I missing something? > > > > > > bpf_helpers.h right now are purely for selftests. But they are really > > > > > > useful outside that context, so I'm making it available for everyone > > > > > > by distributing with libbpf sources. If bpf_helpers.h doesn't work for > > > > > > some specific use case, just don't use it (yet?). > > > > > > > > > > > > I'm still failing to see how it's worse than situation today. > > > > > > > > > > Having a header which works today, but may not work tomorrow is going > > > > > to be pretty bad user experience :( No matter how many warnings you put > > > > > in the source people will get caught off guard by this :( > > > > > > > > > > If you define the current state as "users can use all features of > > > > > libbpf and nothing should break on libbpf update" (which is in my > > > > > understanding a goal of the project, we bent over backwards trying > > > > > to not break things) then adding this header will in fact make things > > > > > worse. The statement in quotes would no longer be true, no? > > > > > > > > So there are few things here. > > > > > > > > 1. About "adding bpf_helpers.h will make things worse". I > > > > categorically disagree, bpf_helpers.h doesn't exist in user land at > > > > all and it's sorely missing. So adding it is strictly better > > > > experience already. Right now people have to re-declare those helper > > > > signatures and do all kinds of unnecessary hackery just to be able to > > > > use BPF stuff, and they still can run into the same problem with > > > > having too old kernel headers. > > > > > > Right, so apps tend to ship their own uapi bpf.h header and helper > > > signatures to avoid these issues. But question becomes once they > > > start using soley bpf_helper.h (also in non-tracing context which > > > is very reasonable to assume), then things might break with the patch > > > as-is once they have a newer libbpf with more signatures than their > > > linux/bpf.h defines (and yes, pulling from GH will have this problem), > > > so we'd need to have an answer to that in order to avoid breaking > > > compilation. > > > > > > [...] > > > > 2. As to the problem of running bleeding-edge libbpf against older > > > > kernel. There are few possible solutions: > > > > > > > > a. we hard-code all those BPF_FUNC_ constants. Super painful and not > > > > nice, but will work. > > > > > > > > b. copy/paste enum bpf_func_id definition into bpf_helpers.h itself > > > > and try to keep it in sync with UAPI. Apart from obvious redundancy > > > > that involves, we also will need to make sure this doesn't conflict > > > > with vmlinux.h, so enum name should be different and each value should > > > > be different (which means some wort of additional prefix or suffix). > > > > > > > > c. BPF UAPI header has __BPF_FUNC_MAPPER macro "iterating" over all > > > > defined values for a particular kernel version. We can use that and > > > > additional macro trickery to conditionally define helpers. Again, we > > > > need to keep in mind that w/ vmlinux.h there is no such macro, so this > > > > should work as well. > > > > > > > > I'm happy to hear opinions about these choices (maybe there are some > > > > other I missed), but in any case I'd like to do it in a follow up > > > > patch and land this one as is. It has already quite a lot packed in > > > > it. I personally lean towards c) as it will have a benefit of not > > > > declaring helpers that are not supported by kernel we are compiling > > > > against, even though it requires additional macro trickery. > > > > > > > > Opinions? > > > > > > Was thinking about something like c) as well. So I tried to do a quick > > > hack. Here is how it could work, but it needs a small change in the > > > __BPF_FUNC_MAPPER(), at least I didn't find an immediate way around it: > > Well, we are stuck with this comma, so rather than have to support two > bpf.h headers, I'd solve the problem for existing one. It's annoying, > but you can do it with having "/* <your macro> /*" in each FN macro err, was supposed to be "*/ (close previous comment) <your macro> /* (open next comment". > and then before you apply everything you add /* and after all the > applications you add */. > > I'm going to prototype something like what you have below, but will > see we I can minimize amount of extra declarations we need. Do you > think this needs to be done as part of this patch set, or I can defer > that to a follow up patch? > > > > > > > static void (*__unspec)(void); > > > static void *(*__map_lookup_elem)(void *map, const void *key); > > > static int (*__map_update_elem)(void *map, const void *key, const void *value, unsigned long long flags); > > > static int (*__map_delete_elem)(void *map, const void *key); > > > static int (*__bpf_probe_read)(void *dst, int size, const void *unsafe_ptr); > > > > > [...]
