On 10/17/19 10:50 AM, Andrii Nakryiko wrote: > On Tue, Oct 15, 2019 at 11:01 PM Andrii Nakryiko <andriin@xxxxxx> wrote: >> >> Define test runner generation meta-rule that codifies dependencies >> between test runner, its tests, and its dependent BPF programs. Use that >> for defining test_progs and test_maps test-runners. Also additionally define >> 2 flavors of test_progs: >> - alu32, which builds BPF programs with 32-bit registers codegen; >> - bpf_gcc, which build BPF programs using GCC, if it supports BPF target. >> >> Overall, this is accomplished through $(eval)'ing a set of generic >> rules, which defines Makefile targets dynamically at runtime. See >> comments explaining the need for 2 $(evals), though. >> >> For each test runner we have (test_maps and test_progs, currently), and, >> optionally, their flavors, the logic of build process is modeled as >> follows (using test_progs as an example): >> - all BPF objects are in progs/: >> - BPF object's .o file is built into output directory from >> corresponding progs/.c file; >> - all BPF objects in progs/*.c depend on all progs/*.h headers; >> - all BPF objects depend on bpf_*.h helpers from libbpf (but not >> libbpf archive). There is an extra rule to trigger bpf_helper_defs.h >> (re-)build, if it's not present/outdated); >> - build recipe for BPF object can be re-defined per test runner/flavor; >> - test files are built from prog_tests/*.c: >> - all such test file objects are built on individual file basis; >> - currently, every single test file depends on all BPF object files; >> this might be improved in follow up patches to do 1-to-1 dependency, >> but allowing to customize this per each individual test; >> - each test runner definition can specify a list of extra .c and .h >> files to be built along test files and test runner binary; all such >> headers are becoming automatic dependency of each test .c file; >> - due to test files sometimes embedding (using .incbin assembly >> directive) contents of some BPF objects at compilation time, which are >> expected to be in CWD of compiler, compilation for test file object does >> cd into test runner's output directory; to support this mode all the >> include paths are turned into absolute paths using $(abspath) make >> function; >> - prog_tests/test.h is automatically (re-)generated with an entry for >> each .c file in prog_tests/; >> - final test runner binary is linked together from test object files and >> extra object files, linking together libbpf's archive as well; >> - it's possible to specify extra "resource" files/targets, which will be >> copied into test runner output directory, if it differes from >> Makefile-wide $(OUTPUT). This is used to ensure btf_dump test cases and >> urandom_read binary is put into a test runner's CWD for tests to find >> them in runtime. >> >> For flavored test runners, their output directory is a subdirectory of >> common Makefile-wide $(OUTPUT) directory with flavor name used as >> subdirectory name. >> >> BPF objects targets might be reused between different test runners, so >> extra checks are employed to not double-define them. Similarly, we have >> redefinition guards for output directories and test headers. >> >> test_verifier follows slightly different patterns and is simple enough >> to not justify generalizing TEST_RUNNER_DEFINE/TEST_RUNNER_DEFINE_RULES >> further to accomodate these differences. Instead, rules for >> test_verifier are minimized and simplified, while preserving correctness >> of dependencies. >> >> Signed-off-by: Andrii Nakryiko <andriin@xxxxxx> >> --- > > BTW, if correctness and DRY-ness argument is not strong enough, these > changes makes clean rebuild from scratch about 2x faster for me: > > BEFORE: `make clean && time make -j50` is 14-15 seconds > AFTER: `make clean && time make -j50` is 7-8 seconds I noticed that too and was about to ask "why?" .. :)
