Andrii Nakryiko wrote: > It's become a common pattern to have a collection of small BPF programs > in one BPF object file, each representing one test case. On user-space > side of such tests we maintain a table of program names and expected > failure or success, along with optional expected verifier log message. > > This works, but each set of tests reimplement this mundane code over and > over again, which is a waste of time for anyone trying to add a new set > of tests. Furthermore, it's quite error prone as it's way too easy to miss > some entries in these manually maintained test tables (as evidences by > dynptr_fail tests, in which ringbuf_release_uninit_dynptr subtest was > accidentally missed; this is fixed in next patch). > > So this patch implements generic verification_tester, which accepts > skeleton name and handles the rest of details: opens and loads BPF > object file, making sure each program is tested in isolation. Optionally > each test case can specify expected BPF verifier log message. In case of > failure, tester makes sure to report verifier log, but it also reports > verifier log in verbose mode unconditionally. > > Now, the interesting deviation from existing custom implementations is > the use of btf_decl_tag attribute to specify expected-to-fail vs > expected-to-succeed markers and, optionally, expected log message > directly next to BPF program source code, eliminating the need to > manually create and update table of tests. > > We define few macros wrapping btf_decl_tag with a convention that all > values of btf_decl_tag start with "comment:" prefix, and then utilizing > a very simple "just_some_text_tag" or "some_key_name=<value>" pattern to > define things like expected success/failure, expected verifier message, > extra verifier log level (if necessary). This approach is demonstrated > by next patch in which two existing sets of failure tests are converted. > > Tester supports both expected-to-fail and expected-to-succeed programs, > though this patch set didn't convert any existing expected-to-succeed > programs yet, as existing tests couple BPF program loading with their > further execution through attach or test_prog_run. One way to allow > testing scenarios like this would be ability to specify custom callback, > executed for each successfully loaded BPF program. This is left for > follow up patches, after some more analysis of existing test cases. > > This verification_tester is, hopefully, a start of a test_verifier > runner, which allows much better "user experience" of defining low-level > verification types that can take advantage of all the libbpf-provided > nicety features on BPF side: global variables, declarative maps, etc. > All while having a choice of defining it in C or as BPF assembly > (through __attribute__((naked)) functions and using embedded asm). This > will be explored in follow up patches as well. > > Signed-off-by: Andrii Nakryiko <andrii@xxxxxxxxxx> > --- LGTM. Acked-by: John Fastabend <john.fastabend@xxxxxxxxx>
