On Mon, Feb 20, 2023 at 3:28 AM Quentin Monnet <quentin@xxxxxxxxxxxxx> wrote: > > 2023-02-17 13:56 UTC-0800 ~ Andrii Nakryiko <andrii.nakryiko@xxxxxxxxx> > > On Fri, Feb 17, 2023 at 3:59 AM Quentin Monnet <quentin@xxxxxxxxxxxxx> wrote: > >> > >> 2023-02-17 11:25 UTC+0100 ~ Puranjay Mohan <puranjay12@xxxxxxxxx> > >>> Hi, > >>> Thanks for the response. > >>> > >>> On Thu, Feb 16, 2023 at 11:13 PM Andrii Nakryiko > >>> <andrii.nakryiko@xxxxxxxxx> wrote: > >>>> > >>>> On Wed, Feb 15, 2023 at 5:48 PM Stanislav Fomichev <sdf@xxxxxxxxxx> wrote: > >>>>> > >>>>> On 02/15, Puranjay Mohan wrote: > >>>>>> The BPF selftests fail to compile on 32-bit architectures as the skeleton > >>>>>> generated by bpftool doesn’t take into consideration the size difference > >>>>>> of > >>>>>> variables on 32-bit/64-bit architectures. > >>>>> > >>>>>> As an example, > >>>>>> If a bpf program has a global variable of type: long, its skeleton will > >>>>>> include > >>>>>> a bss map that will have a field for this variable. The long variable in > >>>>>> BPF is > >>>>>> 64-bit. if we are working on a 32-bit machine, the generated skeleton has > >>>>>> to > >>>>>> compile for that machine where long is 32-bit. > >>>>> > >>>>>> A reproducer for this issue: > >>>>>> root@56ec59aa632f:~# cat test.bpf.c > >>>>>> long var; > >>>>> > >>>>>> root@56ec59aa632f:~# clang -target bpf -g -c test.bpf.c > >>>>> > >>>>>> root@56ec59aa632f:~# bpftool btf dump file test.bpf.o format raw > >>>>>> [1] INT 'long int' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED > >>>>>> [2] VAR 'var' type_id=1, linkage=global > >>>>>> [3] DATASEC '.bss' size=0 vlen=1 > >>>>>> type_id=2 offset=0 size=8 (VAR 'var') > >>>>> > >>>>>> root@56ec59aa632f:~# bpftool gen skeleton test.bpf.o > skeleton.h > >>>>> > >>>>>> root@56ec59aa632f:~# echo "#include \"skeleton.h\"" > test.c > >>>>> > >>>>>> root@56ec59aa632f:~# gcc test.c > >>>>>> In file included from test.c:1: > >>>>>> skeleton.h: In function 'test_bpf__assert': > >>>>>> skeleton.h:231:2: error: static assertion failed: "unexpected > >>>>>> size of \'var\'" > >>>>>> 231 | _Static_assert(sizeof(s->bss->var) == 8, "unexpected > >>>>>> size of 'var'"); > >>>>>> | ^~~~~~~~~~~~~~ > >>>>> > >>>>>> One naive solution for this would be to map ‘long’ to ‘long long’ and > >>>>>> ‘unsigned long’ to ‘unsigned long long’. But this doesn’t solve everything > >>>>>> because this problem is also seen with pointers that are 64-bit in BPF and > >>>>>> 32-bit in 32-bit machines. > >>>>> > >>>>>> I want to work on solving this and am looking for ideas to solve it > >>>>>> efficiently. > >>>>>> The main goal is to make libbbpf/bpftool host architecture agnostic. > >>>>> > >>>>> Looks like bpftool needs to be aware of the target architecture. The > >>>>> same way gcc is doing with build-host-target triplet. I don't > >>>>> think this can be solved with a bunch of typedefs? But I've long > >>>>> forgotten how a pure 32-bit machine looks, so I can't give any > >>>>> useful input :-( > >>>> > >>>> Yeah, I'd rather avoid making bpftool aware of target architecture. > >>>> Three is 32 vs 64 bitness, there is also little/big endianness, etc. > >> > >> I'd rather avoid that too, but for addressing the endianness issue with > >> cross-compiling, reported by Christophe and where the bytecode is not > >> stored with the right endianness in the skeleton file, do you see an > >> alternative? > > > > So bytecode is just a byte array, by itself endianness shouldn't > > matter. The contents of it (ELF itself) is supposed to be of correct > > target endianness, though, right? Or what problem we are talking about > > here? Can you please summarize? > > TL;DR: When cross-compiling, host little-endian bootstrap bpftool cannot > open a big-endian ELF to generate a skeleton from it and build target > big-endian bpftool. > > Long version: Currently, bpftool's Makefile compiles the > skeleton-related programs (skeletons/*.bpf.c) without paying attention > to the target architecture. When cross-compiling, say on a host with LE > for a target with BE, this leads to runtime failure on "bpftool prog > show", because bpftool cannot load the LE bytecode on the BE target > machine. This is Christophe's output in [0]. > > So the first fix is to make the Makefile aware of the target endianness > somehow, and to build this ELF with target endianness. But this is not > enough, because when (host) boostrap bpftool opens the ELF to generate > the skeleton from it before building the final (target) bpftool binary, > then bpf_object__check_endianness() in libbpf refuses to open the ELF if > endianness is not the same as on the host [1]. > > The way I see it, we'd need to make sure libbpf can work with ELFs of a > different endianness -- assuming that's doable -- and to pass it an > option to tell whether LE or BE is expected for a given ELF. Which in > turn would require bootstrap bpftool to be aware of the target > endianness. Or do you see a simpler way? I don't. We might not need to instruct libbpf what endianness should be expected, we can just say that BPF object file with wrong endianness can't be loaded, but could be introspected. So bpf_object__open() should succeed, all the getters and stuff as well. But bpf_object__load() will fail. We already support opening BTF of non-native endianness, that wasn't too bad to support. I'm not sure how much work and extra logic would be necessary for similar cross-endianness support in ELF processing code, though. > > > [0] > https://lore.kernel.org/bpf/0792068b-9aff-d658-5c7d-086e6d394c6c@xxxxxxxxxx/ > [1] > https://lore.kernel.org/bpf/21b09e52-142d-92f5-4f8b-e4190f89383b@xxxxxxxxxx/
