On Wed, Mar 10, 2021 at 2:45 PM Rafael David Tinoco <rafaeldtinoco@xxxxxxxxxx> wrote: > > > > From what I see all the CO-RE relocations applied successfully (even > > though all the offsets stayed the same, so presumably you compiled > > your BPF program with vmlinux.h from the exact same kernel you are > > running it on?). Are you sure that vmlinux image you are providing > > corresponds to the actual kernel you are running on? > > Yep, I have created the following: > > https://pastebin.ubuntu.com/p/h58YyNr4HR/ > Ok, I have no idea, tbh. Maybe `pahole -j` is subtly modifying vmlinux is some way (but then why would kernel start and only fail to load/verify your BPF program?). It's also clear that CO-RE is doing exactly the same instruction patching, so shouldn't be some invalid CO-RE relocation applied, I think. So no idea and not sure how to investigate this. But I think I'd never do `pahole -J` on actual vmlinux image you are going to run. It's much safer and more convenient to make a copy, generate .BTF and then extract just .BTF section into a small binary, which can be provided separately. > to make this easier. > > It is a 4.15.0-1080 kernel and a 4.15.18+. They are pretty close Also this. Seems like it is two different kernels still, however small the difference between them is. Have you tried building *exactly* the same kernel with exactly the same config, but just with pahole -J during the build on vmlinux.o (which is later linked into a final vmlinux) vs running pahole -J on final vmlinux after the build. Notice that two approach differ in terms of which object file is being modified. With link-vmlinux.sh, kernel linker scripts dictate final layout, while if you run `pahole -J` on final vmlinux, it can modify the layout and potentially break something sensitive. pahole is using llvm-objcopy internally, and llvm-objcopy isn't a very trivial tool. > despite the versioning (second was generated with make deb-dpkg). > > I’m using same .config file for both, only difference is that the > 4.15.18+ was compiled with the changed link-vmlinux.sh file. > > The /usr/lib/debug/boot/vmlinux files are generated by the same > build and I have tried 2 or 3 of the existing packaged kernels. The > only thing I did was “pahole -J” to /usr/lib/debug/boot/vmlinux-XXX > files (adding the BTF section to them). > > Running same binary in a 5.8.0-43 kernel works out-of-the-box: > > https://pastebin.ubuntu.com/p/VKTjMcp6Xs/ > > > > > I'd start by comparing libbpf logs for vmlinux you get with modified > > link-vmlinux.sh script and with just explicit pahole -J. If all the > > CO-RE parts are identical, the problem is somewhere else most > > probably. > > The difference between (1) and (2) from the paste (error and success): > > libbpf: CO-RE relocating [0] struct task_struct: found target > candidate [17361] struct task_struct in [vmlinux] > libbpf: CO-RE relocating [0] struct task_struct: found target > candidate [17360] struct task_struct in [vmlinux] > > libbpf: prog 'tcp_connect': relo #0: matching candidate #1 [17361] > struct task_struct.comm (0:90 @ offset 2640) > libbpf: prog 'tcp_connect': relo #0: matching candidate #1 [17360] > struct task_struct.comm (0:90 @ offset 2640) > > Code is at: > > https://github.com/rafaeldtinoco/portablebpf > > and it is not much different than any other libbpf example. > > thanks again for verifying this! > > -rafaeldtinoco > > >
