On 05/08, Stanislav Fomichev wrote:
> On 05/08, Alexei Starovoitov wrote:
> > On Wed, May 08, 2019 at 10:18:41AM -0700, Stanislav Fomichev wrote:
> > > Right now we are not using rcu api correctly: we pass __rcu pointers
> > > to bpf_prog_array_xyz routines but don't use rcu_dereference on them
> > > (see bpf_prog_array_delete_safe and bpf_prog_array_copy in particular).
> > > Instead of sprinkling rcu_dereferences, let's just get rid of those
> > > __rcu annotations and move rcu handling to a higher level.
> > >
> > > It looks like all those routines are called from the rcu update
> > > side and we can use simple rcu_dereference_protected to get a
> > > reference that is valid as long as we hold a mutex (i.e. no other
> > > updater can change the pointer, no need for rcu read section and
> > > there should not be a use-after-free problem).
> > >
> > > To be fair, there is currently no issue with the existing approach
> > > since the calls are mutex-protected, pointer values don't change,
> > > __rcu annotations are ignored. But it's still nice to use proper api.
> > >
> > > The series fixes the following sparse warnings:
> >
> > Absolutely not.
> > please fix it properly.
> > Removing annotations is not a fix.
> I'm fixing it properly by removing the annotations and moving lifetime
> management to the upper layer. See commits 2-4 where I fix the users, the
> first patch is just the "preparation".
>
> The users are supposed to do:
>
> mutex_lock(&x);
> p = rcu_dereference_protected(prog_array, lockdep_is_held(&x))
> // ...
> // call bpf_prog_array helpers while mutex guarantees that
> // the object referenced by p is valid (i.e. no need for bpf_prog_array
> // helpers to care about rcu lifetime)
> // ...
> mutex_unlock(&x);
>
> What am I missing here?
Just to give you my perspective on why current api with __rcu annotations
is working, but not optimal (even if used from the rcu read section).
Sample code:
struct bpf_prog_array __rcu *progs = <comes from somewhere>;
int n;
rcu_read_lock();
n = bpf_prog_array_length(progs);
if (n > 0) {
// do something with __rcu progs
do_something(progs);
}
rcu_read_unlock();
Since progs is __rcu annotated, do_something() would need to do
rcu_dereference again and it might get a different value compared to
whatever bpf_prog_array_free got while doing its dereference.
A better way is not to deal with rcu inside those helpers and let
higher layers do that:
struct bpf_prog_array __rcu *progs = <comes from somewhere>;
struct bpf_prog_array *p;
int n;
rcu_read_lock();
p = rcu_dereference(p);
n = bpf_prog_array_length(p);
if (n > 0) {
do_something(p); // do_something sees the same p as bpf_prog_array_length
}
rcu_read_unlock();
What do you think?
If it sounds reasonable, I can follow up with a v2 because I think I can
replace xchg with rcu_swap_protected as well. Or I can resend for bpf-next to
have another round of discussion. Thoughts?
