On Mon, Dec 11, 2023 at 4:49 AM Sergei Trofimovich <slyich@xxxxxxxxx> wrote:
>
> Hi bpf@!
>
> Tl;DR:
>
> BPF programs built with `clang`'s `-g -Wa,--compress-debug-sections`
> fail when processed by `bpftool` on section alignment checks as:
>
> $ cat src/core/bpf/socket_bind/bpf.c
> struct socket_bind_map_t { int value; };
> struct socket_bind_map_t sd_bind_allow __attribute__((section(".maps"), used));
>
> $ clang-16 -target bpf -fno-stack-protector -c src/core/bpf/socket_bind/bpf.c -o bpf.unstripped.o -g -Wa,--compress-debug-sections
> $ bpftool gen object bpf.o bpf.unstripped.o
> libbpf: ELF section #9 has inconsistent alignment addr=8 != d=4 in bpf.unstripped.o
So we are talking about this check in libbpf's linker:
if (sec->shdr->sh_addralign != sec->data->d_align)
And it's confusing that it would be ok to have actual data alignment
that doesn't match declared alignment in section header? Is that
expected in ELF in general?
Having said that, I think we can mitigate all this easily by ignoring
DWARF sections. Libbpf linker drops all that while linking, see
is_ignored_sec() check, but when doing a sanity check we still
validate everything for DWARF sections.
So perhaps an easy way out is just to ignore DWARF sections in sanity
checks as well?
> Error: failed to link 'bpf.unstripped.o': Invalid argument (22)
>
> This happens only when `bpftool` is a 32-bit ELF binary. 64-bit
> `bpftool` seems to work as expected.
>
> More words:
>
> Here is my understanding of the failure. Without the
> `-Wa,--compress-debug-sections` option `clang` generates byte-aligned
> uncompressed `.debug` sections:
>
> $ clang -target bpf -fno-stack-protector -c src/core/bpf/socket_bind/bpf.c -o bpf.unstripped.o -g
> $ readelf -SW bpf.unstripped.o
> There are 19 section headers, starting at offset 0x530:
>
> Section Headers:
> [Nr] Name Type Address Off Size ES Flg Lk Inf Al
> [ 0] NULL 0000000000000000 000000 000000 00 0 0 0
> [ 1] .strtab STRTAB 0000000000000000 000471 0000b8 00 0 0 1
> [ 2] .text PROGBITS 0000000000000000 000040 000000 00 AX 0 0 4
> [ 3] .maps PROGBITS 0000000000000000 000040 000004 00 WA 0 0 4
> [ 4] .debug_abbrev PROGBITS 0000000000000000 000044 00004c 00 0 0 1
> [ 5] .debug_info PROGBITS 0000000000000000 000090 00003d 00 0 0 1
> [ 6] .rel.debug_info REL 0000000000000000 000380 000040 10 I 18 5 8
> [ 7] .debug_str_offsets PROGBITS 0000000000000000 0000cd 000024 00 0 0 1
> [ 8] .rel.debug_str_offsets REL 0000000000000000 0003c0 000070 10 I 18 7 8
> [ 9] .debug_str PROGBITS 0000000000000000 0000f1 000080 01 MS 0 0 1
> [10] .debug_addr PROGBITS 0000000000000000 000171 000010 00 0 0 1
> [11] .rel.debug_addr REL 0000000000000000 000430 000010 10 I 18 10 8
> [12] .BTF PROGBITS 0000000000000000 000184 000099 00 0 0 4
> [13] .rel.BTF REL 0000000000000000 000440 000010 10 I 18 12 8
> [14] .debug_line PROGBITS 0000000000000000 00021d 000043 00 0 0 1
> [15] .rel.debug_line REL 0000000000000000 000450 000020 10 I 18 14 8
> [16] .debug_line_str PROGBITS 0000000000000000 000260 000041 01 MS 0 0 1
> [17] .llvm_addrsig LOOS+0xfff4c03 0000000000000000 000470 000001 00 E 18 0 1
> [18] .symtab SYMTAB 0000000000000000 0002a8 0000d8 18 1 8 8
> Key to Flags:
> W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
> L (link order), O (extra OS processing required), G (group), T (TLS),
> C (compressed), x (unknown), o (OS specific), E (exclude),
> D (mbind), p (processor specific)
>
> Note how `.debug_str` has `alignment=1` here. Loading of such a section
> has no problems on x86_64 or i686.
>
> But when we add `-Wa,--compress-debug-sections` some debug sections
> become aligned 8 bytes (usual `ELF64`) alignment:
>
> $ clang -target bpf -fno-stack-protector -c src/core/bpf/socket_bind/bpf.c -o bpf.unstripped.o -g -Wa,--compress-debug-sections
> $ LANG=C readelf -SW bpf.unstripped.o
> There are 19 section headers, starting at offset 0x530:
>
> Section Headers:
> [Nr] Name Type Address Off Size ES Flg Lk Inf Al
> [ 0] NULL 0000000000000000 000000 000000 00 0 0 0
> [ 1] .strtab STRTAB 0000000000000000 000471 0000b8 00 0 0 1
> [ 2] .text PROGBITS 0000000000000000 000040 000000 00 AX 0 0 4
> [ 3] .maps PROGBITS 0000000000000000 000040 000004 00 WA 0 0 4
> [ 4] .debug_abbrev PROGBITS 0000000000000000 000044 00004c 00 0 0 1
> [ 5] .debug_info PROGBITS 0000000000000000 000090 00003d 00 0 0 1
> [ 6] .rel.debug_info REL 0000000000000000 000380 000040 10 I 18 5 8
> [ 7] .debug_str_offsets PROGBITS 0000000000000000 0000cd 000024 00 0 0 1
> [ 8] .rel.debug_str_offsets REL 0000000000000000 0003c0 000070 10 I 18 7 8
> [ 9] .debug_str PROGBITS 0000000000000000 0000f1 000086 01 MSC 0 0 8
> [10] .debug_addr PROGBITS 0000000000000000 000177 000010 00 0 0 1
> [11] .rel.debug_addr REL 0000000000000000 000430 000010 10 I 18 10 8
> [12] .BTF PROGBITS 0000000000000000 000188 000099 00 0 0 4
> [13] .rel.BTF REL 0000000000000000 000440 000010 10 I 18 12 8
> [14] .debug_line PROGBITS 0000000000000000 000221 000043 00 0 0 1
> [15] .rel.debug_line REL 0000000000000000 000450 000020 10 I 18 14 8
> [16] .debug_line_str PROGBITS 0000000000000000 000264 000041 01 MS 0 0 1
> [17] .llvm_addrsig LOOS+0xfff4c03 0000000000000000 000470 000001 00 E 18 0 1
> [18] .symtab SYMTAB 0000000000000000 0002a8 0000d8 18 1 8 8
> Key to Flags:
> W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
> L (link order), O (extra OS processing required), G (group), T (TLS),
> C (compressed), x (unknown), o (OS specific), E (exclude),
> D (mbind), p (processor specific)
>
> Note how `.debug_str` alignment changed from 1-byte to 8-byte alignment.
>
> AFAIU 8-byte alignment of compressed section for `ELF64` files is expected:
> compressed sections all have a small header that consistes of a few
> 64-bit values (compression type, uncompressed size, uncompressed data
> alignment).
>
> Thus the binary loads as expected by `x86_64` `bpftool` and fails when
> loaded by `i686` `bpftool`.
>
> Should `bpftool` work in this scenario? Or compressed sections are not
> supported on 32-bit hosts?
>
> It feels like debugging sections with strings should be easily
> decompressable on any host type.
>
> Thanks!
>
> --
>
> Sergei
>
