Re: [PATCH bpf-next] selftests/bpf: use auto-dependencies for test objects

[Eduard Zingerman <eddyz87@xxxxxxxxx>]

On Thu, 2024-07-11 at 21:09 +0000, Ihor Solodrai wrote:


[...]

> @@ -583,14 +596,20 @@ endif
>  # Note: we cd into output directory to ensure embedded BPF object is found
>  $(TRUNNER_TEST_OBJS): $(TRUNNER_OUTPUT)/%.test.o:			\
>  		      $(TRUNNER_TESTS_DIR)/%.c				\
> -		      $(TRUNNER_EXTRA_HDRS)				\
> -		      $(TRUNNER_BPF_OBJS)				\
> -		      $(TRUNNER_BPF_SKELS)				\
> -		      $(TRUNNER_BPF_LSKELS)				\
> -		      $(TRUNNER_BPF_SKELS_LINKED)			\
> -		      $$(BPFOBJ) | $(TRUNNER_OUTPUT)
> +		      $(TRUNNER_OUTPUT)/%.test.d
>  	$$(call msg,TEST-OBJ,$(TRUNNER_BINARY),$$@)
> -	$(Q)cd $$(@D) && $$(CC) -I. $$(CFLAGS) -c $(CURDIR)/$$< $$(LDLIBS) -o $$(@F)
> +	$(Q)cd $$(@D) && $$(CC) -I. $$(CFLAGS) -MMD -MT $$@ -c $(CURDIR)/$$< $$(LDLIBS) -o $$(@F)

Hi Ihor, nice patch, thank you for working on this!

While re-testing the patch I've noticed a strange behaviour:
$ cd <kernel>/tools/testing/selftests/bpf
$ git clean -xfd .
$ make -j14
$ touch progs/atomics.c 
$ make -j14 test_progs
  CLNG-BPF [test_maps] atomics.bpf.o
  CLNG-BPF [test_maps] atomics.bpf.o
  CLNG-BPF [test_maps] atomics.bpf.o
  GEN-SKEL [test_progs] atomics.lskel.h
  GEN-SKEL [test_progs-cpuv4] atomics.lskel.h
  GEN-SKEL [test_progs-no_alu32] atomics.lskel.h
  TEST-OBJ [test_progs] arena_htab.test.o
  TEST-OBJ [test_progs] atomics.test.o
  ... many lines ...
  TEST-OBJ [test_progs] uprobe_multi_test.test.o
  TEST-OBJ [test_progs] usdt.test.o
  TEST-OBJ [test_progs] verify_pkcs7_sig.test.o
  TEST-OBJ [test_progs] vmlinux.test.o
  TEST-OBJ [test_progs] xdp_adjust_tail.test.o
  TEST-OBJ [test_progs] xdp_metadata.test.o
  TEST-OBJ [test_progs] xdp_synproxy.test.o
  BINARY   test_progs
16:15:34 bpf$ make -j14 test_progs
  TEST-OBJ [test_progs] bpf_iter.test.o
  TEST-OBJ [test_progs] bpf_nf.test.o
  TEST-OBJ [test_progs] bpf_obj_id.test.o
  TEST-OBJ [test_progs] btf.test.o
  TEST-OBJ [test_progs] btf_write.test.o
  TEST-OBJ [test_progs] cgrp_local_storage.test.o
  TEST-OBJ [test_progs] iters.test.o
  TEST-OBJ [test_progs] lsm_cgroup.test.o
  TEST-OBJ [test_progs] send_signal.test.o
  TEST-OBJ [test_progs] sockmap_basic.test.o
  TEST-OBJ [test_progs] sockmap_listen.test.o
  TEST-OBJ [test_progs] trace_vprintk.test.o
  TEST-OBJ [test_progs] usdt.test.o
  TEST-OBJ [test_progs] xdp_metadata.test.o
  BINARY   test_progs
16:15:37 bpf$ make -j14 test_progs
  TEST-OBJ [test_progs] bpf_obj_id.test.o
  TEST-OBJ [test_progs] sockmap_listen.test.o
  TEST-OBJ [test_progs] xdp_metadata.test.o
  BINARY   test_progs
16:15:39 bpf$ make -j14 test_progs
  TEST-OBJ [test_progs] sockmap_listen.test.o
  BINARY   test_progs
16:15:41 bpf$ make -j14 test_progs
make: 'test_progs' is up to date.

Interestingly enough, this does not happen with your off-list version of
the patch shared this morning, the one with order-only dependency for .d:

  +$(TRUNNER_TEST_OBJS): $(TRUNNER_OUTPUT)/%.test.o: $(TRUNNER_TESTS_DIR)/%.c | $(TRUNNER_OUTPUT)/%.test.d

Could you please double-check what's going on?

> +
> +$(TRUNNER_TEST_OBJS:.o=.d): $(TRUNNER_OUTPUT)/%.test.d:			\
> +			    $(TRUNNER_TESTS_DIR)/%.c			\
> +			    $(TRUNNER_EXTRA_HDRS)			\
> +			    $(TRUNNER_BPF_SKELS)			\
> +			    $(TRUNNER_BPF_LSKELS)			\
> +			    $(TRUNNER_BPF_SKELS_LINKED)			\
> +			    $$(BPFOBJ) | $(TRUNNER_OUTPUT)
> +
> +include $(wildcard $(TRUNNER_TEST_OBJS:.o=.d))
> +
>  
>  $(TRUNNER_EXTRA_OBJS): $(TRUNNER_OUTPUT)/%.o:				\
>  		       %.c						\

[...]