On Sat, Aug 31, 2024 at 11:18 AM Ihor Solodrai <ihor.solodrai@xxxxx> wrote: > > Andrii, Eduard, > > On Friday, August 30th, 2024 at 1:34 PM, Andrii Nakryiko <andrii.nakryiko@xxxxxxxxx> wrote: > > [...] > > > I've applied patches as is, despite them not solving the issue > > completely, as they are moving us in the right direction anyways. I do > > get slightly different BTF every single time I rebuild my kernel, so > > the change in patch #2 doesn't yet help me. > > Thanks for applying the patches. > I didn't realize vmlinux.h generation is non-deterministic. Interesting. > > > > > For libbpf headers, Ihor, can you please follow up with adding > > bpf_helper_defs.h as a dependency? > > I've tried tracking down where bpf_helper_defs.h is coming from and > (assuming my analysis is correct) this header is generated by > `scripts/bpf_doc.py`. From the selftests/bpf point of view the > dependency chain is as follows: > > 1. vmlinux.h depends on bpftool: > > $(INCLUDE_DIR)/vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL) | $(INCLUDE_DIR) > > 2. bpftool is installed for selftests via `make -C tools/bpf/bpftool install-bin`: > > BPFTOOL ?= $(DEFAULT_BPFTOOL) > $(DEFAULT_BPFTOOL): ... > $(Q)$(MAKE) $(submake_extras) -C $(BPFTOOLDIR) ... install-bin > > 3. bpftool install-bin depends on libbpf: > > $(OUTPUT)bpftool: $(OBJS) $(LIBBPF) > ... > install-bin: $(OUTPUT)bpftool > > > 4. $(LIBBPF) recipe runs `make -C tools/lib/bpf install_headers`, > which depends on $(BPF_GENERATED) which equals to $(BPF_HELPER_DEFS) > > BPF_GENERATED := $(BPF_HELPER_DEFS) > ... > install_headers: $(BPF_GENERATED) $(INSTALL_SRC_HDRS) $(INSTALL_GEN_HDRS) > > 5. Finally $(BPF_HELPER_DEFS) recipe executes the python script (in lib/bpf): > > $(BPF_HELPER_DEFS): $(srctree)/tools/include/uapi/linux/bpf.h > $(QUIET_GEN)$(srctree)/scripts/bpf_doc.py --header \ > --file $(srctree)/tools/include/uapi/linux/bpf.h > $(BPF_HELPER_DEFS) > > > I don't see any benefit to adding bpf_helper_defs.h as a direct > dependency of anything in selftests/bpf. %.bpf.o already depend on > vmlinux.h, and unless we somehow get rid of vmlinux.h dependency on > bpftool, bpf_helper_defs.h should always be there at a point when > %.bpf.o objects are compiled. > Making sure that bpf_helper_defs.h is generated by the time .bpf.o is being compiled is one thing. Triggering .bpf.o regeneration when bpf_helper_defs.h changes is another. The second used to be important when we were adding new helpers, otherwise we can get compilation error because of missing helper definitions. This is much less of an issue today, we we might just leave it as is. Making sure bpf_helper_defs.h is there might be good enough. > > > > > I have some ideas on how to make BTF regeneration in vmlinux.h itself > > unnecessary, that might help with this issue. Separately (depending on > > what are the negatives of the reproducible_build option) we can look > > into making pahole have more consistent internal BTF type ordering > > without negatively affecting the overall BTF dedup performance in > > pahole. Hopefully I can work with Ihor on this as follow ups. > > I still know little about how all this machinery works, but I'd be > glad to help. I'd like to avoid regenerating BTF inside the vmlinux image, if possible. This will cut down significantly on incremental kernel build times. We can chat about this separately a bit later, don't worry. > > > > > P.S. I also spent more time than I'm willing to admit trying to > > improve bpftool's BTF sorting to minimize the chance of vmlinux.h > > contents being different, and I think I removed a bunch of cases where > > we had unnecessary differences, but still, it's fundamentally > > non-deterministic to do everything based on type and field names, > > unfortunately. > > [...] > >
