As an extension of [1], allow bpf_fastcall patterns for kfuncs:
- pattern rules are the same as for helpers;
- spill/fill removal is allowed only for kfuncs listed in the
is_fastcall_kfunc_call (under assumption that such kfuncs would
always be members of special_kfunc_list).
Allow bpf_fastcall rewrite for bpf_cast_to_kern_ctx() and
bpf_rdonly_cast() in order to conjure selftests for this feature.
After this patch-set verifier would rewrite the program below:
r2 = 1
*(u64 *)(r10 - 32) = r2
call %[bpf_cast_to_kern_ctx]
r2 = *(u64 *)(r10 - 32)
r0 = r2;"
As follows:
r2 = 1 /* spill/fill at r10[-32] is removed */
r0 = r1 /* replacement for bpf_cast_to_kern_ctx() */
r0 = r2
exit
Also, attribute used by LLVM implementation of the feature had been
changed from no_caller_saved_registers to bpf_fastcall (see [2]).
This patch-set replaces references to nocsr by references to
bpf_fastcall to keep LLVM and Kernel parts in sync.
[1] no_caller_saved_registers attribute for helper calls
https://lore.kernel.org/bpf/20240722233844.1406874-1-eddyz87@xxxxxxxxx/
[2] [BPF] introduce __attribute__((bpf_fastcall))
https://github.com/llvm/llvm-project/pull/105417
Changes v2->v3:
- added a patch fixing arch_mask handling in test_loader,
otherwise newly added tests for the feature were skipped
(a fix for regression introduced by a recent commit);
- fixed warning regarding unused 'params' variable;
- applied stylistical fixes suggested by Yonghong;
- added acks from Yonghong;
Changes v1->v2:
- added two patches replacing all mentions of nocsr by bpf_fastcall
(suggested by Andrii);
- removed KF_NOCSR flag (suggested by Yonghong).
v1: https://lore.kernel.org/bpf/20240812234356.2089263-1-eddyz87@xxxxxxxxx/
v2: https://lore.kernel.org/bpf/20240817015140.1039351-1-eddyz87@xxxxxxxxx/
Eduard Zingerman (6):
bpf: rename nocsr -> bpf_fastcall in verifier
selftests/bpf: rename nocsr -> bpf_fastcall in selftests
bpf: support bpf_fastcall patterns for kfuncs
bpf: allow bpf_fastcall for bpf_cast_to_kern_ctx and bpf_rdonly_cast
selftests/bpf: by default use arch mask allowing all archs
selftests/bpf: check if bpf_fastcall is recognized for kfuncs
include/linux/bpf.h | 6 +-
include/linux/bpf_verifier.h | 18 +-
kernel/bpf/helpers.c | 2 +-
kernel/bpf/verifier.c | 181 +++++++++++-------
.../selftests/bpf/prog_tests/verifier.c | 4 +-
...rifier_nocsr.c => verifier_bpf_fastcall.c} | 81 ++++++--
tools/testing/selftests/bpf/test_loader.c | 2 +-
7 files changed, 192 insertions(+), 102 deletions(-)
rename tools/testing/selftests/bpf/progs/{verifier_nocsr.c => verifier_bpf_fastcall.c} (89%)
--
2.45.2
