On Wed, Feb 2, 2022 at 9:59 AM Timo Beckers <timo@xxxxxxxxxx> wrote:
>
> On 2/2/22 08:17, Yonghong Song wrote:
> >
> >
> > On 2/1/22 10:07 AM, Vincent Li wrote:
> >> On Fri, Jan 28, 2022 at 10:27 AM Vincent Li <vincent.mc.li@xxxxxxxxx> wrote:
> >>>
> >>> On Thu, Jan 27, 2022 at 5:50 PM Yonghong Song <yhs@xxxxxx> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On 1/25/22 12:32 PM, Vincent Li wrote:
> >>>>> On Tue, Jan 25, 2022 at 9:52 AM Vincent Li <vincent.mc.li@xxxxxxxxx> wrote:
> >>>>>>
> >>>>>> this is macro I suspected in my implementation that could cause issue with BTF
> >>>>>>
> >>>>>> #define ENABLE_VTEP 1
> >>>>>> #define VTEP_ENDPOINT (__u32[]){0xec48a90a, 0xee48a90a, 0x1f48a90a,
> >>>>>> 0x2048a90a, }
> >>>>>> #define VTEP_MAC (__u64[]){0x562e984c3682, 0x582e984c3682,
> >>>>>> 0x5eaaed93fdf2, 0x5faaed93fdf2, }
> >>>>>> #define VTEP_NUMS 4
> >>>>>>
> >>>>>> On Tue, Jan 25, 2022 at 9:38 AM Vincent Li <vincent.mc.li@xxxxxxxxx> wrote:
> >>>>>>>
> >>>>>>> Hi
> >>>>>>>
> >>>>>>> While developing Cilium VTEP integration feature
> >>>>>>> https://github.com/cilium/cilium/pull/17370, I found a strange issue
> >>>>>>> that seems related to BTF and probably caused by my specific
> >>>>>>> implementation, the issue is described in
> >>>>>>> https://github.com/cilium/cilium/issues/18616, I don't know much about
> >>>>>>> BTF and not sure if my implementation is seriously flawed or just some
> >>>>>>> implementation bug or maybe not compatible with BTF. Strangely, the
> >>>>>>> issue appears related to number of VTEPs I use, no problem with 1 or 2
> >>>>>>> VTEP, 3, 4 VTEPs will have problem with BTF, any guidance from BTF
> >>>>>>> experts are appreciated :-).
> >>>>>>>
> >>>>>>> Thanks
> >>>>>>>
> >>>>>>> Vincent
> >>>>>
> >>>>> Sorry for previous top post
> >>>>>
> >>>>> it looks the compiler compiles the cilium bpf_lxc.c to bpf_lxc.o
> >>>>> differently and added " [21] .rodata.cst32 PROGBITS
> >>>>> 0000000000000000 00011e68" when following macro exceeded 2 members
> >>>>>
> >>>>> #define VTEP_ENDPOINT (__u32[]){0xec48a90a, 0xee48a90a, 0x1f48a90a,
> >>>>> 0x2048a90a, }
> >>>>>
> >>>>> no ".rodata.cst32" compiled in bpf_lxc.o when above VTEP_ENDPOINT
> >>>>> member <=2. any reason why compiler would do that?
> >>>>
> >>>> Regarding to why compiler generates .rodata.cst32, the reason is
> >>>> you have some 32-byte constants which needs to be saved somewhere.
> >>>> For example,
> >>>>
> >>>> $ cat t.c
> >>>> struct t {
> >>>> long c[2];
> >>>> int d[4];
> >>>> };
> >>>> struct t g;
> >>>> int test()
> >>>> {
> >>>> struct t tmp = {.c = {1, 2}, .d = {3, 4}};
> >>>> g = tmp;
> >>>> return 0;
> >>>> }
> >>>>
> >>>> $ clang -target bpf -O2 -c t.c
> >>>> $ llvm-readelf -S t.o
> >>>> ...
> >>>> [ 4] .rodata.cst32 PROGBITS 0000000000000000 0000a8 000020
> >>>> 20 AM 0 0 8
> >>>> ...
> >>>>
> >>>> In the above code, if you change the struct size, say from 32 bytes to
> >>>> 40 bytes, the rodata.cst32 will go away.
> >>>
> >>> Thanks Yonghong! I guess it is cilium/ebpf needs to recognize rodata.cst32 then
> >>
> >> Hi Yonghong,
> >>
> >> Here is a follow-up question, it looks cilium/ebpf parse vmlinux and
> >> stores BTF type info in btf.Spec.namedTypes, but the elf object file
> >> provided by user may have section like rodata.cst32 generated by
> >> compiler that does not have accompanying BTF type info stored in
> >> btf.Spec.NamedTypes for the rodata.cst32, how vmlinux can be
> >> guaranteed to have every BTF type info from application/user provided
> >> elf object file ? I guess there is no guarantee.
> >
> > vmlinux holds kernel types. rodata.cst32 holds data. If the type of
> > rodata.cst32 needs to be emitted, the type will be encoded in bpf
> > program BTF.
> >
> > Did you actually find an issue with .rodata.cst32 section? Such a
> > section is typically generated by the compiler for initial data
> > inside the function and llvm bpf backend tries to inline the
> > values through a bunch of load instructions. So even you see
> > .rodata.cst32, typically you can safely ignore it.
> >
> >>
> >> Vincent
> >
>
> Hi Yonghong,
>
> Thanks for the reproducer. Couldn't figure out what to do with .rodata.cst32,
> since there are no symbols and no BTF info for that section.
>
> The values found in .rodata.cst32 are indeed inlined in the bytecode as you
> mentioned, so it seems like we can ignore it.
>
> Why does the compiler emit these sections? cilium/ebpf assumed up until now
> that all sections starting with '.rodata' are datasecs and must be loaded into
> the kernel, which of course needs accompanying BTF.
It doesn't need an accompanying BTF. libbpf loads all .rodata.*
sections (unless BPF program code doesn't reference it at all, which
is mostly for backwards compatibility with old kernels that don't
support global variables) and it all works good.
>
> What other .rodata.* should we expect?
>
I'd suggest instead of whitelisting all possible .rodata.* section
names just allow any such data section. So that users can have their
own custom .rodata (and .data) sections as well.
> Thanks,
>
> Timo
