On 7/12/24 6:36 AM, Ihor Solodrai wrote:
Make use of -M compiler options when building .test.o objects to generate .d files and avoid re-building all tests every time. Previously, if a single test bpf program under selftests/bpf/progs/*.c has changed, make would rebuild all the *.bpf.o, *.skel.h and *.test.o objects, which is a lot of unnecessary work. A typical dependency chain is: progs/x.c -> x.bpf.o -> x.skel.h -> x.test.o -> trunner_binary However for many tests it's not a 1:1 mapping by name, and so far %.test.o have been simply dependent on all %.skel.h files, and %.skel.h files on all %.bpf.o objects. Avoid full rebuilds by instructing the compiler (via -MMD) to produce *.d files with real dependencies, and appropriately including them. Exploit make feature that rebuilds included makefiles if they were changed by setting %.test.d as prerequisite for %.test.o files. A couple of examples of compilation time speedup (after the first clean build): $ touch progs/verifier_and.c && time make -j8 Before: real 0m16.651s After: real 0m2.245s $ touch progs/read_vsyscall.c && time make -j8 Before: real 0m15.743s After: real 0m1.575s A drawback of this change is that now there is an overhead due to make processing lots of .d files, which potentially may slow down unrelated targets. However a time to make all from scratch hasn't changed significantly: $ make clean && time make -j8 Before: real 1m31.148s After: real 1m30.309s Suggested-by: Eduard Zingerman <eddyz87@xxxxxxxxx> Signed-off-by: Ihor Solodrai <ihor.solodrai@xxxxx> --- v1 -> v2: Make %.test.d prerequisite order only
Looks like BPF CI trips over this and various tests are failing : https://github.com/kernel-patches/bpf/actions/runs/9902529566/job/27356664649 [...] Tests exit status: 1 Notice: Success: 538/3821, Skipped: 62, Failed: 5 Error: #66 core_reloc Error: #66/4 core_reloc/flavors Error: #66/4 core_reloc/flavors run_core_reloc_tests:FAIL:btf_src_file unexpected error: -1 (errno 2) Error: #66/5 core_reloc/flavors__err_wrong_name Error: #66/5 core_reloc/flavors__err_wrong_name run_core_reloc_tests:FAIL:btf_src_file unexpected error: -1 (errno 2) Error: #66/6 core_reloc/nesting Error: #66/6 core_reloc/nesting run_core_reloc_tests:FAIL:btf_src_file unexpected error: -1 (errno 2) Error: #66/7 core_reloc/nesting___anon_embed Error: #66/8 core_reloc/nesting___struct_union_mixup Error: #66/9 core_reloc/nesting___extra_nesting [...]
