On Wed, Apr 13, 2022 at 12:52:53PM -0700, Andrii Nakryiko wrote:
> On Wed, Apr 13, 2022 at 12:39 PM <sdf@xxxxxxxxxx> wrote:
> >
> > On 04/13, Andrii Nakryiko wrote:
> > > On Wed, Apr 13, 2022 at 11:33 AM Stanislav Fomichev <sdf@xxxxxxxxxx>
> > > wrote:
> > > >
> > > > Commit 7d08c2c91171 ("bpf: Refactor BPF_PROG_RUN_ARRAY family of macros
> > > > into functions") switched a bunch of BPF_PROG_RUN macros to inline
> > > > routines. This changed the semantic a bit. Due to arguments expansion
> > > > of macros, it used to be:
> > > >
> > > > rcu_read_lock();
> > > > array = rcu_dereference(cgrp->bpf.effective[atype]);
> > > > ...
> > > >
> > > > Now, with with inline routines, we have:
> > > > array_rcu = rcu_dereference(cgrp->bpf.effective[atype]);
> > > > /* array_rcu can be kfree'd here */
> > > > rcu_read_lock();
> > > > array = rcu_dereference(array_rcu);
> > > >
> >
> > > So subtle difference, wow...
> >
> > > But this open-coding of rcu_read_lock() seems very unfortunate as
> > > well. Would making BPF_PROG_RUN_ARRAY back to a macro which only does
> > > rcu lock/unlock and grabs effective array and then calls static inline
> > > function be a viable solution?
> >
> > > #define BPF_PROG_RUN_ARRAY_CG_FLAGS(array_rcu, ctx, run_prog, ret_flags) \
> > > ({
> > > int ret;
> >
> > > rcu_read_lock();
> > > ret =
> > > __BPF_PROG_RUN_ARRAY_CG_FLAGS(rcu_dereference(array_rcu), ....);
> > > rcu_read_unlock();
> > > ret;
> > > })
> >
> >
> > > where __BPF_PROG_RUN_ARRAY_CG_FLAGS is what
> > > BPF_PROG_RUN_ARRAY_CG_FLAGS is today but with __rcu annotation dropped
> > > (and no internal rcu stuff)?
> >
> > Yeah, that should work. But why do you think it's better to hide them?
> > I find those automatic rcu locks deep in the call stack a bit obscure
> > (when reasoning about sleepable vs non-sleepable contexts/bpf).
> >
> > I, as the caller, know that the effective array is rcu-managed (it
> > has __rcu annotation) and it seems natural for me to grab rcu lock
> > while work with it; I might grab it for some other things like cgroup
> > anyway.
>
> If you think that having this more explicitly is better, I'm fine with
> that as well. I thought a simpler invocation pattern would be good,
> given we call bpf_prog_run_array variants in quite a lot of places. So
> count me indifferent. I'm curious what others think.
Would it work if the bpf_prog_run_array_cg() directly takes the
'struct cgroup *cgrp' argument instead of the array ?
bpf_prog_run_array_cg() should know what protection is needed
to get member from the cgrp ptr. The sk call path should be able
to provide a cgrp ptr. For current cgrp, pass NULL as the cgrp
pointer and then current will be used in bpf_prog_run_array_cg().
A rcu_read_lock() is needed anyway to get the current's cgrp
and can be done together in bpf_prog_run_array_cg().
That there are only two remaining bpf_prog_run_array() usages
from lirc and bpf_trace which are not too bad to have them
directly do rcu_read_lock on their own struct ?
