On Wed, Feb 28, 2024 at 2:49 AM Alexei Starovoitov <alexei.starovoitov@xxxxxxxxx> wrote: > > On Tue, Feb 27, 2024 at 8:51 AM Benjamin Tissoires > <benjamin.tissoires@xxxxxxxxxx> wrote: > > > > On Tue, Feb 27, 2024 at 5:36 PM Eduard Zingerman <eddyz87@xxxxxxxxx> wrote: > > > > > > On Tue, 2024-02-27 at 17:18 +0100, Benjamin Tissoires wrote: > > > [...] > > > > > > > Hmm, I must still be missing a piece of the puzzle: > > > > if I declare bpf_timer_set_sleepable_cb() to take a third "aux" > > > > argument, given that it is declared as kfunc, I also must declare it in > > > > my bpf program, or I get the following: > > > > > > > > # libbpf: extern (func ksym) 'bpf_timer_set_sleepable_cb': func_proto [264] incompatible with vmlinux [18151] > > > > > > > > And if I declare it, then I don't know what to pass, given that this is > > > > purely added by the verifier: > > > > > > > > 43: (85) call bpf_timer_set_sleepable_cb#18152 > > > > arg#2 pointer type STRUCT bpf_prog_aux must point to scalar, or struct with scalar > > > > > > Right, something has to be done about number of arguments and we don't > > > have a convenient mechanism for this afaik. > > > > > > The simplest way would be to have two kfuncs: > > > - one with 2 arguments, used form bpf program; > > > - another with 3 arguments, used at runtime; > > > - replace former by latter during rewrite. > > > > It's hacky but seems interesting enough to be tested :) > > Too hacky imo :) > > Let's follow the existing pattern. > See: > __bpf_kfunc void *bpf_obj_new_impl(u64 local_type_id__k, void *meta__ign) > > __ign suffix tells the verifier to ignore it. > > Then we do: > #define bpf_obj_new(type) \ > ((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL)) > > and later the verifier replaces arg2 with the correct pointer. \o/ Thanks, it works :) > > > We also could use the suffix (like __uninit, __k, etc...), but it > > might introduce more headaches than the 2 kfuncs you are proposing. > > Only one kfunc pls. Let's not make it more complex than necessary. > > We cannot easily add a suffix to tell libbpf to ignore that arg, > since bpf_core_types_are_compat() compares types and there are > no argument names in the types. > So it will be a significant surgery for libbpf to find the arg name > in vmlinux BTF and strcmp the suffix. Yeah, I guessed so. Having a single #define is fine, especially given that there are already a lot of them for the same purpose. > > > > > > > > > Could you please provide more details on what exactly it complains about? > > > > > > > Well, there is a simple reason: that code is never reached because, in > > that function, there is a `if (insn->src_reg == > > BPF_PSEUDO_KFUNC_CALL)` above that unconditionally terminates with a > > `continue`. So basically this part of the code is never hit. > > > > I'll include that new third argument and the dual kfunc call in > > fixup_kfunc_call() and report if it works from here. > > Something is wrong. fixup_kfunc_call() can rewrite args with whatever > it wants. > Are you sure you've added bpf_timer_set_sleepable_cb to special_kfunc_list ? > Yeah, but as I mentioned, I wasn't hacking at the correct place. I was not doing the changes in the fixup_kfunc_call() but in the helper processing, so that path was not hit. But with your instructions it works. I have a couple of changes to do and the selftests to add and the series will be ready. Cheers, Benjamin
