Andrii Nakryiko <andrii.nakryiko@xxxxxxxxx> wrote:
On Wed, Mar 10, 2021 at 2:45 PM Rafael David Tinoco
<rafaeldtinoco@xxxxxxxxxx> wrote:
From what I see all the CO-RE relocations applied successfully (even
though all the offsets stayed the same, so presumably you compiled
your BPF program with vmlinux.h from the exact same kernel you are
running it on?). Are you sure that vmlinux image you are providing
corresponds to the actual kernel you are running on?
Yep, I have created the following:
https://pastebin.ubuntu.com/p/h58YyNr4HR/
Ok, I have no idea, tbh. Maybe `pahole -j` is subtly modifying vmlinux
is some way (but then why would kernel start and only fail to
load/verify your BPF program?). It's also clear that CO-RE is doing
exactly the same instruction patching, so shouldn't be some invalid
CO-RE relocation applied, I think. So no idea and not sure how to
investigate this.
But I think I'd never do `pahole -J` on actual vmlinux image you are
going to run. It's much safer and more convenient to make a copy,
generate .BTF and then extract just .BTF section into a small binary,
which can be provided separately.
FOUND the cause of the issue...
Compiling the EXACT same kernel with different building scripts
(deb-pkg vs debian/rules in my case) resulted in a very similar
kernel (same .config, same autoconf.h, no visible changes).
Unfortunately one of the kernels worked fined (reading the BTF
extracted section OR same section within a vmlinux entire
file).. but the other did not.
Instrumenting this bad 4.15 kernel (out of debian/rules build)
I found that the following sanity checks took place in kernel:
bpf_check():
if (log->len_total < 128 || log->len_total > UINT_MAX >> 8 || !log->level
|| !log->ubuf)
and a simple change in libbpf (mitigation of course):
attr.log_buf = 0;
attr.log_level = 0;
attr.log_size = 0;
before
fd = sys_bpf_prog_load(&attr, sizeof(attr));
made the code to also run in this second kernel (built with the
debian/rules building scripts):
https://pastebin.ubuntu.com/p/scJDM3D9Zr/
Now I still have to discover why log_buf is miss-behaving in this
kernel being built with debian/rules* scripts and not with the
vanilla building scripts (despite config file being the same).
(FYIO, documenting it here for others also)...
