Hi Shung-Hsi, On Thu, 13 Apr 2023 at 10:23, Shung-Hsi Yu <shung-hsi.yu@xxxxxxxx> wrote: > > Hi, > > I'm considering switch to bpftool's mirror on GitHub for packaging (instead > of using the source found in kernel), but realize that it should goes > hand-in-hand with how libbpf is packaged, which eventually leads these > questions: > > What is the suggested approach for packaging bpftool and libbpf? > Which source is preferred, GitHub or kernel? As you can see from the previous discussions, the suggested approach would be to package from the GitHub mirror, with libbpf and bpftool in sync. My main argument for the mirror is that it keeps things simpler, and there's no need to deal with the rest of the kernel sources for these packages. Download from the mirrors, build, ship. But then I have limited experience at packaging for distros, and I can understand Toke's point of view, too. So ultimately, the call is yours. > Does bpftool work on older kernel? It should, although it's not perfect. Most features from current bpftool should work as expected on older kernels. However, if I remember correctly you would have trouble loading programs on pre-BTF kernels, because bpftool relies on libbpf >= 1.0 and only accepts map definitions with BTF info, and attempts to create these maps with BTF, which fails and blocks the load process. But we're trying to keep backward-compatibility, so if we're only talking of kernels recent enough to support BTF, then I'd expect bpftool to work. If this is not the case, please report on this list. > > Our current approach is that we (openSUSE/SLES) essentially have two version > of libbpf: a public shared library that uses GitHub mirror as source, which > the general userspace sees and links to; and a private static library built > from kernel source used by bpftool, perf, resolve_btfids, selftests, etc. > A survey of other distros (Arch, Debian, Fedora, Ubuntu) suggest that they > took similar approach. I would like them to reconsider this choice eventually. Sounds like for RHEL, this will be a tough sell :). At least, I'd love Ubuntu to have a real bpftool package instead of having to install linux-tools-common + linux-tools-generic, or to have distros in general (Ubuntu/Debian at least) stop compiling out the JIT disassembler, although this is not strictly related to the location of the sources. I've not found the time to reach out to package maintainers yet. > > This approach means that the version of bpftool and libbpf are _not_ always > in sync[1], which I read may causes problem since libbpf and bpftool depends > on specific version of each other[2]. Whatever source you use, I would strongly recommend finding a way to keep both in sync. Libbpf has stabilised its API when reaching 1.0, but bpftool taps into some of the internals of the library. Features or new definitions are usually added at the same time to libbpf and bpftool, and if you get a mismatch between the two, you're taking risks to get build issues. > > Using the GitHub mirror of bpftool to package both libbpf and bpftool would > kept their version in sync, and was suggested[3]. Although the same could be > said if we switch back to packaging libbpf from kernel source, an additional > appeal for using GitHub mirrors is that it decouples bpftool from kernel, > making it easily upgradable and with a clearer changelog (the latter is > quite important for enterprise users) like libbpf. Happy to read these changelogs I write are useful to someone :). Yes, this is my point. > > The main concern with using GitHub mirror is that bpftool may be updated far > beyond the version that comes with the runtime kernel. AFAIK bpftool should > work on older kernel since CO-RE is used for built-in BPF iterators and the > underlying libbpf work on older kernel itself. Nonetheless, it would be nice > to get a confirmation from the maintainers. As explained above - Mostly, it should work. Otherwise, we can look into fixing it. As a side note, I'm open to suggestions/contributions to make life easier for packaging for the mirror. For example, Mahé and I recently added GitHub workflows to ship statically-built binaries for amd64 and arm64 on releases, as well as tarballs with both bpftool+libbpf sources. If there's something else to make packaging easier, I'm happy to talk about it. > > Are there any other downsides to switching to GitHub mirror for bpftool? Nothing else comes to mind, but again I'm not packaging for distros. See Toke's message. I hope this helps, Quentin
