Andrii, Eduard,
On Friday, August 30th, 2024 at 1:34 PM, Andrii Nakryiko <andrii.nakryiko@xxxxxxxxx> wrote:
[...]
> I've applied patches as is, despite them not solving the issue
> completely, as they are moving us in the right direction anyways. I do
> get slightly different BTF every single time I rebuild my kernel, so
> the change in patch #2 doesn't yet help me.
Thanks for applying the patches.
I didn't realize vmlinux.h generation is non-deterministic. Interesting.
>
> For libbpf headers, Ihor, can you please follow up with adding
> bpf_helper_defs.h as a dependency?
I've tried tracking down where bpf_helper_defs.h is coming from and
(assuming my analysis is correct) this header is generated by
`scripts/bpf_doc.py`. From the selftests/bpf point of view the
dependency chain is as follows:
1. vmlinux.h depends on bpftool:
$(INCLUDE_DIR)/vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL) | $(INCLUDE_DIR)
2. bpftool is installed for selftests via `make -C tools/bpf/bpftool install-bin`:
BPFTOOL ?= $(DEFAULT_BPFTOOL)
$(DEFAULT_BPFTOOL): ...
$(Q)$(MAKE) $(submake_extras) -C $(BPFTOOLDIR) ... install-bin
3. bpftool install-bin depends on libbpf:
$(OUTPUT)bpftool: $(OBJS) $(LIBBPF)
...
install-bin: $(OUTPUT)bpftool
4. $(LIBBPF) recipe runs `make -C tools/lib/bpf install_headers`,
which depends on $(BPF_GENERATED) which equals to $(BPF_HELPER_DEFS)
BPF_GENERATED := $(BPF_HELPER_DEFS)
...
install_headers: $(BPF_GENERATED) $(INSTALL_SRC_HDRS) $(INSTALL_GEN_HDRS)
5. Finally $(BPF_HELPER_DEFS) recipe executes the python script (in lib/bpf):
$(BPF_HELPER_DEFS): $(srctree)/tools/include/uapi/linux/bpf.h
$(QUIET_GEN)$(srctree)/scripts/bpf_doc.py --header \
--file $(srctree)/tools/include/uapi/linux/bpf.h > $(BPF_HELPER_DEFS)
I don't see any benefit to adding bpf_helper_defs.h as a direct
dependency of anything in selftests/bpf. %.bpf.o already depend on
vmlinux.h, and unless we somehow get rid of vmlinux.h dependency on
bpftool, bpf_helper_defs.h should always be there at a point when
%.bpf.o objects are compiled.
>
> I have some ideas on how to make BTF regeneration in vmlinux.h itself
> unnecessary, that might help with this issue. Separately (depending on
> what are the negatives of the reproducible_build option) we can look
> into making pahole have more consistent internal BTF type ordering
> without negatively affecting the overall BTF dedup performance in
> pahole. Hopefully I can work with Ihor on this as follow ups.
I still know little about how all this machinery works, but I'd be
glad to help.
>
> P.S. I also spent more time than I'm willing to admit trying to
> improve bpftool's BTF sorting to minimize the chance of vmlinux.h
> contents being different, and I think I removed a bunch of cases where
> we had unnecessary differences, but still, it's fundamentally
> non-deterministic to do everything based on type and field names,
> unfortunately.
[...]
