On 8/28/22 1:53 AM, KP Singh wrote:
On Sat, Aug 27, 2022 at 1:15 AM Andrii Nakryiko <andrii@xxxxxxxxxx> wrote:
Add a small tool, veristat, that allows mass-verification of
a set of *libbpf-compatible* BPF ELF object files. For each such object
file, veristat will attempt to verify each BPF program *individually*.
Regardless of success or failure, it parses BPF verifier stats and
outputs them in human-readable table format. In the future we can also
add CSV and JSON output for more scriptable post-processing, if necessary.
veristat allows to specify a set of stats that should be output and
ordering between multiple objects and files (e.g., so that one can
easily order by total instructions processed, instead of default file
name, prog name, verdict, total instructions order).
This tool should be useful for validating various BPF verifier changes
or even validating different kernel versions for regressions.
Cool stuff!
+1, out of curiosity, did you try with different kernels to see the deltas?
I think this would be useful for cases beyond these (i.e. for users to get
stats about the verifier in general) and it's worth thinking if this should
be built into bpftool?
Here's an example for some of the heaviest selftests/bpf BPF object
files:
$ sudo ./veristat -s insns,file,prog {pyperf,loop,test_verif_scale,strobemeta,test_cls_redirect,profiler}*.linked3.o
File Program Verdict Duration, us Total insns Total states Peak states
------------------------------------ ------------------------------------ ------- ------------ ----------- ------------ -----------
loop3.linked3.o while_true failure 350990 1000001 9663 9663
[...]
nit: Looks like CI on gcc is bailing:
https://github.com/kernel-patches/bpf/runs/8072477251?check_suite_focus=true
[...]
INSTALL /tmp/work/bpf/bpf/tools/testing/selftests/bpf/tools/include/bpf/skel_internal.h
In file included from /tmp/work/bpf/bpf/tools/testing/selftests/bpf/tools/include/bpf/libbpf.h:20,
INSTALL /tmp/work/bpf/bpf/tools/testing/selftests/bpf/tools/include/bpf/libbpf_version.h
from veristat.c:17:
/tmp/work/bpf/bpf/tools/testing/selftests/bpf/tools/include/bpf/libbpf_common.h:13:10: fatal error: libbpf_version.h: No such file or directory
13 | #include "libbpf_version.h"
| ^~~~~~~~~~~~~~~~~~
compilation terminated.
INSTALL /tmp/work/bpf/bpf/tools/testing/selftests/bpf/tools/include/bpf/usdt.bpf.h
HOSTCC /tmp/work/bpf/bpf/tools/testing/selftests/bpf/tools/build/libbpf/fixdep.o
make: *** [Makefile:165: /tmp/work/bpf/bpf/tools/testing/selftests/bpf/veristat.o] Error 1
make: *** Waiting for unfinished jobs....
I wonder, to detect regressions in pruning behavior, could we add a test_progs subtest to load
selected obj files and compare before/after 'verified insns' numbers? The workflow probably
makes this a bit hard to run with a kernel before this change, but maybe it could be a starting
point where we have a checked-in file containing current numbers, and e.g. if the new change
crosses a threshold of current +10% then the test could fail?
Thanks,
Daniel
