2023-02-17 13:56 UTC-0800 ~ Andrii Nakryiko <andrii.nakryiko@xxxxxxxxx> > On Fri, Feb 17, 2023 at 3:59 AM Quentin Monnet <quentin@xxxxxxxxxxxxx> wrote: >> >> 2023-02-17 11:25 UTC+0100 ~ Puranjay Mohan <puranjay12@xxxxxxxxx> >>> Hi, >>> Thanks for the response. >>> >>> On Thu, Feb 16, 2023 at 11:13 PM Andrii Nakryiko >>> <andrii.nakryiko@xxxxxxxxx> wrote: >>>> >>>> On Wed, Feb 15, 2023 at 5:48 PM Stanislav Fomichev <sdf@xxxxxxxxxx> wrote: >>>>> >>>>> On 02/15, Puranjay Mohan wrote: >>>>>> The BPF selftests fail to compile on 32-bit architectures as the skeleton >>>>>> generated by bpftool doesn’t take into consideration the size difference >>>>>> of >>>>>> variables on 32-bit/64-bit architectures. >>>>> >>>>>> As an example, >>>>>> If a bpf program has a global variable of type: long, its skeleton will >>>>>> include >>>>>> a bss map that will have a field for this variable. The long variable in >>>>>> BPF is >>>>>> 64-bit. if we are working on a 32-bit machine, the generated skeleton has >>>>>> to >>>>>> compile for that machine where long is 32-bit. >>>>> >>>>>> A reproducer for this issue: >>>>>> root@56ec59aa632f:~# cat test.bpf.c >>>>>> long var; >>>>> >>>>>> root@56ec59aa632f:~# clang -target bpf -g -c test.bpf.c >>>>> >>>>>> root@56ec59aa632f:~# bpftool btf dump file test.bpf.o format raw >>>>>> [1] INT 'long int' size=8 bits_offset=0 nr_bits=64 encoding=SIGNED >>>>>> [2] VAR 'var' type_id=1, linkage=global >>>>>> [3] DATASEC '.bss' size=0 vlen=1 >>>>>> type_id=2 offset=0 size=8 (VAR 'var') >>>>> >>>>>> root@56ec59aa632f:~# bpftool gen skeleton test.bpf.o > skeleton.h >>>>> >>>>>> root@56ec59aa632f:~# echo "#include \"skeleton.h\"" > test.c >>>>> >>>>>> root@56ec59aa632f:~# gcc test.c >>>>>> In file included from test.c:1: >>>>>> skeleton.h: In function 'test_bpf__assert': >>>>>> skeleton.h:231:2: error: static assertion failed: "unexpected >>>>>> size of \'var\'" >>>>>> 231 | _Static_assert(sizeof(s->bss->var) == 8, "unexpected >>>>>> size of 'var'"); >>>>>> | ^~~~~~~~~~~~~~ >>>>> >>>>>> One naive solution for this would be to map ‘long’ to ‘long long’ and >>>>>> ‘unsigned long’ to ‘unsigned long long’. But this doesn’t solve everything >>>>>> because this problem is also seen with pointers that are 64-bit in BPF and >>>>>> 32-bit in 32-bit machines. >>>>> >>>>>> I want to work on solving this and am looking for ideas to solve it >>>>>> efficiently. >>>>>> The main goal is to make libbbpf/bpftool host architecture agnostic. >>>>> >>>>> Looks like bpftool needs to be aware of the target architecture. The >>>>> same way gcc is doing with build-host-target triplet. I don't >>>>> think this can be solved with a bunch of typedefs? But I've long >>>>> forgotten how a pure 32-bit machine looks, so I can't give any >>>>> useful input :-( >>>> >>>> Yeah, I'd rather avoid making bpftool aware of target architecture. >>>> Three is 32 vs 64 bitness, there is also little/big endianness, etc. >> >> I'd rather avoid that too, but for addressing the endianness issue with >> cross-compiling, reported by Christophe and where the bytecode is not >> stored with the right endianness in the skeleton file, do you see an >> alternative? > > So bytecode is just a byte array, by itself endianness shouldn't > matter. The contents of it (ELF itself) is supposed to be of correct > target endianness, though, right? Or what problem we are talking about > here? Can you please summarize? TL;DR: When cross-compiling, host little-endian bootstrap bpftool cannot open a big-endian ELF to generate a skeleton from it and build target big-endian bpftool. Long version: Currently, bpftool's Makefile compiles the skeleton-related programs (skeletons/*.bpf.c) without paying attention to the target architecture. When cross-compiling, say on a host with LE for a target with BE, this leads to runtime failure on "bpftool prog show", because bpftool cannot load the LE bytecode on the BE target machine. This is Christophe's output in [0]. So the first fix is to make the Makefile aware of the target endianness somehow, and to build this ELF with target endianness. But this is not enough, because when (host) boostrap bpftool opens the ELF to generate the skeleton from it before building the final (target) bpftool binary, then bpf_object__check_endianness() in libbpf refuses to open the ELF if endianness is not the same as on the host [1]. The way I see it, we'd need to make sure libbpf can work with ELFs of a different endianness -- assuming that's doable -- and to pass it an option to tell whether LE or BE is expected for a given ELF. Which in turn would require bootstrap bpftool to be aware of the target endianness. Or do you see a simpler way? [0] https://lore.kernel.org/bpf/0792068b-9aff-d658-5c7d-086e6d394c6c@xxxxxxxxxx/ [1] https://lore.kernel.org/bpf/21b09e52-142d-92f5-4f8b-e4190f89383b@xxxxxxxxxx/
