Hi, I'm considering switch to bpftool's mirror on GitHub for packaging (instead of using the source found in kernel), but realize that it should goes hand-in-hand with how libbpf is packaged, which eventually leads these questions: What is the suggested approach for packaging bpftool and libbpf? Which source is preferred, GitHub or kernel? Does bpftool work on older kernel? Our current approach is that we (openSUSE/SLES) essentially have two version of libbpf: a public shared library that uses GitHub mirror as source, which the general userspace sees and links to; and a private static library built from kernel source used by bpftool, perf, resolve_btfids, selftests, etc. A survey of other distros (Arch, Debian, Fedora, Ubuntu) suggest that they took similar approach. This approach means that the version of bpftool and libbpf are _not_ always in sync[1], which I read may causes problem since libbpf and bpftool depends on specific version of each other[2]. Using the GitHub mirror of bpftool to package both libbpf and bpftool would kept their version in sync, and was suggested[3]. Although the same could be said if we switch back to packaging libbpf from kernel source, an additional appeal for using GitHub mirrors is that it decouples bpftool from kernel, making it easily upgradable and with a clearer changelog (the latter is quite important for enterprise users) like libbpf. The main concern with using GitHub mirror is that bpftool may be updated far beyond the version that comes with the runtime kernel. AFAIK bpftool should work on older kernel since CO-RE is used for built-in BPF iterators and the underlying libbpf work on older kernel itself. Nonetheless, it would be nice to get a confirmation from the maintainers. Are there any other downsides to switching to GitHub mirror for bpftool? A side note: if we want all userspace visible libbpf to have a coherent version, perf needs to use the shared libbpf library as well (either autodetected or forced with LIBBPF_DYNAMIC=1 like Fedora[4]). But having to backport patches to kernel source to keep up with userspace package (libbpf) changes could be a pain. Shung-Hsi 1: In practice kernel & bpftool version often falls behind libbpf in snapshot distro (e.g. openSUSE Leap, Ubutnu LTS), and the other way around on rolling distro (e.g. openSUSE Tumbleweed, Arch). 2: https://lore.kernel.org/bpf/CAADnVQK-arrrNrgtu48_f--WCwR5ki2KGaX=mN2qmW_AcRyb=w@xxxxxxxxxxxxxx/ 3: https://lore.kernel.org/bpf/20191204233948.opvlopjkxe5o66lr@xxxxxxxxxxxxxxxxxxxxxxxxxxxx/ 4: https://src.fedoraproject.org/rpms/kernel-tools/blob/82960989c918f81fcc6742a6d765afbec5fa4f74/f/kernel-tools.spec#_248
