Em Wed, Apr 19, 2023 at 12:42:39PM -0700, Andrii Nakryiko escreveu: > On Wed, Apr 19, 2023 at 3:55 AM Shung-Hsi Yu <shung-hsi.yu@xxxxxxxx> wrote: > > > > Thanks for sharing! I though I'd expands on what you said to draw a clearer > > picture of the challenges. > > > > On Thu, Apr 13, 2023 at 01:00:20PM +0200, Toke Høiland-Jørgensen wrote: > > > Shung-Hsi Yu <shung-hsi.yu@xxxxxxxx> writes: > > > > > > > A side note: if we want all userspace visible libbpf to have a coherent > > > > version, perf needs to use the shared libbpf library as well (either > > > > autodetected or forced with LIBBPF_DYNAMIC=1 like Fedora[4]). But having to > > > > backport patches to kernel source to keep up with userspace package (libbpf) > > > > changes could be a pain. > > > > Here some more context for completeness. Kernel source changes are published > > at a much slower pace than userspace. When an application in the kernel > > source (e.g. perf) depends on the userspace library, it's kind of like > > trying to catchup a car on a bike, which is doable, as evident by the That is why perf is continuously (at least) build tested against lots of distros, see the last test output: [perfbuilder@five ~]$ export BUILD_TARBALL=http://192.168.86.10/perf/perf-6.3.0-rc1.tar.xz [perfbuilder@five ~]$ time dm 1 151.45 almalinux:8 : Ok gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-16) , clang version 14.0.6 (Red Hat 14.0.6-1.module_el8.7.0+3277+b822483f) 2 150.54 almalinux:9 : Ok gcc (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2) , clang version 14.0.6 (Red Hat 14.0.6-4.el9_1) 3 159.04 alpine:3.15 : Ok gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027 , Alpine clang version 12.0.1 4 153.81 alpine:3.16 : Ok gcc (Alpine 11.2.1_git20220219) 11.2.1 20220219 , Alpine clang version 13.0.1 5 137.88 alpine:3.17 : Ok gcc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924 , Alpine clang version 15.0.7 6 139.32 alpine:edge : Ok gcc (Alpine 12.2.1_git20220924-r9) 12.2.1 20220924 , Alpine clang version 16.0.0 7 109.51 alt:p9 : Ok x86_64-alt-linux-gcc (GCC) 8.4.1 20200305 (ALT p9 8.4.1-alt0.p9.1) , clang version 10.0.0 8 104.99 alt:p10 : Ok x86_64-alt-linux-gcc (GCC) 10.3.1 20210703 (ALT Sisyphus 10.3.1-alt2) , clang version 11.0.1 9 85.35 alt:sisyphus : Ok x86_64-alt-linux-gcc (GCC) 12.1.1 20220518 (ALT Sisyphus 12.1.1-alt2) , ALT Linux Team clang version 13.0.1 10 111.22 amazonlinux:2 : Ok gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-15) , clang version 11.1.0 (Amazon Linux 2 11.1.0-1.amzn2.0.2) 11 127.05 amazonlinux:2023 : Ok gcc (GCC) 11.3.1 20221121 (Red Hat 11.3.1-4) , clang version 15.0.6 (Amazon Linux 15.0.6-3.amzn2023.0.2) 12 126.56 amazonlinux:devel : Ok gcc (GCC) 11.3.1 20221121 (Red Hat 11.3.1-4) , clang version 15.0.6 (Amazon Linux 15.0.6-3.amzn2023.0.2) 13 154.76 archlinux:base : Ok gcc (GCC) 12.2.0 , clang version 14.0.6 14 135.39 centos:stream : Ok gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-18) , clang version 15.0.7 (Red Hat 15.0.7-1.module_el8.8.0+1258+af79b238) 15 40.01 clearlinux:latest : Ok gcc (Clear Linux OS for Intel Architecture) 12.2.1 20230412 releases/gcc-12.2.0-699-g43ab94d20e 16 95.89 debian:10 : Ok gcc (Debian 8.3.0-6) 8.3.0 , Debian clang version 11.0.1-2~deb10u1 17 121.95 debian:11 : Ok gcc (Debian 10.2.1-6) 10.2.1 20210110 , Debian clang version 11.0.1-2 18 136.49 debian:experimental : Ok gcc (Debian 12.2.0-14) 12.2.0 , Debian clang version 14.0.6 19 29.78 debian:experimental-x-arm64 : Ok aarch64-linux-gnu-gcc (Debian 12.2.0-14) 12.2.0 20 22.96 debian:experimental-x-mips : Ok mips-linux-gnu-gcc (Debian 12.2.0-14) 12.2.0 21 3.37 debian:experimental-x-mips64 : FAIL gcc version 10.2.1 20210110 (Debian 10.2.1-6) 22 11.29 debian:experimental-x-mipsel : FAIL gcc version 12.2.0 (Debian 12.2.0-14) 23 33.28 fedora:26 : Ok gcc (GCC) 7.3.1 20180130 (Red Hat 7.3.1-2) 24 33.29 fedora:27 : Ok gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-6) 25 29.77 fedora:28 : Ok gcc (GCC) 8.3.1 20190223 (Red Hat 8.3.1-2) 26 32.27 fedora:29 : Ok gcc (GCC) 8.3.1 20190223 (Red Hat 8.3.1-2) 27 34.18 fedora:30 : Ok gcc (GCC) 9.3.1 20200408 (Red Hat 9.3.1-2) 28 139.98 fedora:31 : Ok gcc (GCC) 9.3.1 20200408 (Red Hat 9.3.1-2) , clang version 9.0.1 (Fedora 9.0.1-4.fc31) 29 119.93 fedora:32 : Ok gcc (GCC) 10.3.1 20210422 (Red Hat 10.3.1-1) , clang version 10.0.1 (Fedora 10.0.1-3.fc32) 30 123.87 fedora:33 : Ok gcc (GCC) 10.3.1 20210422 (Red Hat 10.3.1-1) , clang version 11.0.0 (Fedora 11.0.0-3.fc33) 31 188.77 fedora:34 : Ok gcc (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2) , clang version 12.0.1 (Fedora 12.0.1-1.fc34) 32 22.44 fedora:34-x-ARC-glibc : Ok arc-linux-gcc (ARC HS GNU/Linux glibc toolchain 2019.03-rc1) 8.3.1 20190225 33 20.33 fedora:34-x-ARC-uClibc : Ok arc-linux-gcc (ARCv2 ISA Linux uClibc toolchain 2019.03-rc1) 8.3.1 20190225 34 172.51 fedora:35 : Ok gcc (GCC) 11.3.1 20220421 (Red Hat 11.3.1-3) , clang version 13.0.1 (Fedora 13.0.1-1.fc35) 35 138.29 fedora:36 : Ok gcc (GCC) 12.2.1 20221121 (Red Hat 12.2.1-4) , clang version 14.0.5 (Fedora 14.0.5-2.fc36) 36 139.99 fedora:37 : Ok gcc (GCC) 12.2.1 20221121 (Red Hat 12.2.1-4) , clang version 15.0.7 (Fedora 15.0.7-1.fc37) 37 144.73 fedora:38 : Ok gcc (GCC) 13.0.1 20230401 (Red Hat 13.0.1-0) , clang version 16.0.0 (Fedora 16.0.0-2.fc38) 38 191.37 fedora:39 : Ok gcc (GCC) 13.0.1 20230404 (Red Hat 13.0.1-0) , clang version 16.0.1 (Fedora 16.0.1-1.fc39) 39 187.13 fedora:rawhide : Ok gcc (GCC) 13.0.1 20230404 (Red Hat 13.0.1-0) , clang version 16.0.1 (Fedora 16.0.1-1.fc39) 40 13.08 gentoo:stage3 : FAIL gcc version 12.2.1 20230121 (Gentoo 12.2.1_p20230121-r1 p10) 41 152.25 manjaro:base : Ok gcc (GCC) 12.2.1 20230201 , clang version 15.0.7 42 148.30 opensuse:15.4 : Ok gcc (SUSE Linux) 7.5.0 , clang version 13.0.1 43 152.82 opensuse:15.5 : Ok gcc (SUSE Linux) 7.5.0 , clang version 15.0.7 44 187.75 opensuse:tumbleweed : Ok gcc (SUSE Linux) 12.2.1 20221020 [revision 0aaef83351473e8f4eb774f8f999bbe87a4866d7] , clang version 15.0.6 45 133.67 oraclelinux:8 : Ok gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-16.0.2) , clang version 14.0.6 (Red Hat 14.0.6-1.0.1.module+el8.7.0+20823+214a699d) 46 132.26 oraclelinux:9 : Ok gcc (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2.1.0.2) , clang version 14.0.6 (Red Hat 14.0.6-4.0.1.el9_1) 47 150.02 rockylinux:8 : Ok gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-16) , clang version 14.0.6 (Red Hat 14.0.6-1.module+el8.7.0+1080+d88dc670) 48 133.78 rockylinux:9 : Ok gcc (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2) , clang version 14.0.6 (Red Hat 14.0.6-4.el9_1) 49 28.97 ubuntu:18.04 : Ok gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 50 25.06 ubuntu:18.04-x-arm : Ok arm-linux-gnueabihf-gcc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0 51 25.06 ubuntu:18.04-x-arm64 : Ok aarch64-linux-gnu-gcc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0 52 20.54 ubuntu:18.04-x-m68k : Ok m68k-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 53 24.56 ubuntu:18.04-x-powerpc : Ok powerpc-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 54 26.07 ubuntu:18.04-x-powerpc64 : Ok powerpc64-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 55 26.46 ubuntu:18.04-x-powerpc64el : Ok powerpc64le-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 56 113.72 ubuntu:18.04-x-riscv64 : Ok riscv64-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 57 22.55 ubuntu:18.04-x-s390 : Ok s390x-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 58 24.35 ubuntu:18.04-x-sh4 : Ok sh4-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 59 22.54 ubuntu:18.04-x-sparc64 : Ok sparc64-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 60 31.87 ubuntu:20.04 : Ok gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0 61 179.64 ubuntu:22.04 : Ok gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0 , Ubuntu clang version 14.0.0-1ubuntu1 62 1.26 ubuntu:22.04-x-riscv64 : FAIL gcc version 11.3.0 (Ubuntu 11.3.0-1ubuntu1~22.04) 63 87.25 ubuntu:22.10 : Ok gcc (Ubuntu 12.2.0-3ubuntu1) 12.2.0 , Ubuntu clang version 15.0.7 64 86.96 ubuntu:23.04 : Ok gcc (Ubuntu 12.2.0-17ubuntu1) 12.2.0 , Ubuntu clang version 15.0.7 BUILD_TARBALL_HEAD=f8b04f975d2c3d7c8e8cb53155744c20a41813ac 65 6011.52 real 101m23.391s user 0m51.216s sys 0m40.407s [perfbuilder@five ~]$ > > plethora of userspace libraries perf already depends on. While I don't > > having experience maintaining perf, judging by tools/perf/Makefile.config > > that does not seem like an easy feat. Not that bad having the tests we have in place :-) > > For perf to use libbpf in kernel would mean that it's just depending on > > something that moves at the same pace. More tests to check build both with the in-kernel libbpf and with the one in the distro: ⬢[acme@toolbox perf-tools-next]$ grep LIBBPF_DYNAMIC tools/perf/tests/make make_libbpf_dynamic := LIBBPF_DYNAMIC=1 ⬢[acme@toolbox perf-tools-next]$ > > That said, maybe perf won't need additional backport to keep up with libbpf > > as long as we keep it within that same major version (and disable > > deprecation warning)? @Andrii > > > > Now that We've got pass libbpf 1.0 it seems like a good time to reconsider. > > I'm not sure what the proposal is, but I'll delegate to Arnaldo. > > > So basically, this here is the reason we're building libbpf from the > > > kernel tree for the RHEL package: If we use the github version we'd need > > > to juggle two different versions of libbpf, one for the in-kernel-tree > > > users (perf as you mention, but also the BPF selftests), and one for the Have you tried with LIBBPF_DYNAMIC=1? > > > userspace packages. Also, having libbpf in the kernel tree means we can > > > just backport patches to it along with the BPF-related kernel patches > > > (we do quite extensive BPF backports for each RHEL version). > > > > > Finally, building from the kernel tree means we can use the existing > > > kernel-related procedures for any out of order hotfixes (since AFAIK none > > > of the github repositories have any concept of stable branches that > > > receive fixes). > > > > +1 > > > > Got something similar in place as well and being able to stick with existing > > procedure is appealing. > > > > > YMMV of course, but figured I'd share our reasoning. To be clear, > > > building from the kernel tree is not without its own pain points (mostly > > > related to how the build scripts are structured for our kernel builds). > > > We've discussed moving to the github version of libbpf multiple times, > > > but every time we've concluded that it would be more, not less, painful > > > than having the kernel tree be the single source of truth. > > > > We package maintainer are certainly quite hard to please :) > > > > Just having an individual package easy to work with is not enough, we want > > it to be easier for most packages before jumping on the bandwagon, which is > > why this email ended up talking about perf despite it started as a > > discussion on packaging libbpf and bpftool. > > > > I suppose the mileage depends on the build system & scripts in use and how > > much backporting is done; the more kernel backporting (along with more > > established processes in place), the more painful it'd be to move to the > > GitHub version. My gut feeling is that SLES do less backporting compared to > > RHEL when it comes to BPF, and that probably placed us closer to the middle > > ground. > > Even though libbpf source is developed in kernel repo, it's not tied > to specific kernel. So any kernel backports have no relevance to > libbpf itself. It's yet another reason to switch to Github mirror. > Github is merging libbpf-related fixes from both bpf and bpf-next > trees during sync, and is meant to always be the latest and best > version with all fixes included. > > I won't claim anything for perf, maybe Arnaldo can clarify, but I > suspect that perf is also meant to be relatively independent from > specific kernel and work on wide variety of kernels. > > As for stable branches. For libbpf, we don't have it because we didn't > need it yet. We did have bug fix patch releases that seem to be > working out fine, though. > > > > > Thanks, > > Shung-Hsi > > > > > -Toke > > > -- - Arnaldo
