On Fri, May 29, 2020 at 01:25:06AM -0700, Andrii Nakryiko wrote: > > index 11584618e861..26b18b6a3dbc 100644 > > --- a/kernel/bpf/arraymap.c > > +++ b/kernel/bpf/arraymap.c > > @@ -393,6 +393,11 @@ static void array_map_free(struct bpf_map *map) > > */ > > synchronize_rcu(); > > > > + /* arrays could have been used by both sleepable and non-sleepable bpf > > + * progs. Make sure to wait for both prog types to finish executing. > > + */ > > + synchronize_srcu(&bpf_srcu); > > + > > to minimize churn later on when you switch to rcu_trace, maybe extract > synchronize_rcu() + synchronize_srcu(&bpf_srcu) into a function (e.g., > something like synchronize_sleepable_bpf?), exposed as an internal > API? That way you also wouldn't need to add bpf_srcu to linux/bpf.h? I think the opposite is must have actually. I think rcu operations should never be hidden in helpers. All rcu/srcu/rcu_trace ops should always be open coded. > > @@ -577,8 +577,8 @@ static void *__htab_map_lookup_elem(struct bpf_map *map, void *key) > > struct htab_elem *l; > > u32 hash, key_size; > > > > - /* Must be called with rcu_read_lock. */ > > - WARN_ON_ONCE(!rcu_read_lock_held()); > > + /* Must be called with s?rcu_read_lock. */ > > + WARN_ON_ONCE(!rcu_read_lock_held() && !srcu_read_lock_held(&bpf_srcu)); > > > > Similar to above, might be worthwhile extracting into a function? This one I'm 50/50, since this pattern will be in many places. But what kind of helper that would be? Clear name is very hard. WARN_ON_ONCE(!bpf_specific_rcu_lock_held()) ? Moving WARN into the helper would be even worse. When rcu_trace is available the churn of patches to convert srcu to rcu_trace will be a good thing. The patches will convey the difference. Like bpf_srcu will disappear. They will give a way to do benchmarking before/after and will help to go back to srcu in unlikely case there is some obscure bug in rcu_trace. Hiding srcu vs rcu_trace details behind helpers is not how the code should read. The trade off with one and another will be different case by case. Like synchronize_srcu() is ok, but synchronize_rcu_trace() may be too heavy in the trampoline update code and extra counter would be needed. Also there will be synchronize_multi() that I plan to use as well. > > > > + if (prog->aux->sleepable && prog->type != BPF_PROG_TYPE_TRACING && > > + prog->type != BPF_PROG_TYPE_LSM) { > > + verbose(env, "Only fentry/fexit/fmod_ret and lsm programs can be sleepable\n"); > > + return -EINVAL; > > + } > > > BPF_PROG_TYPE_TRACING also includes iterator and raw tracepoint > programs. You mention only fentry/fexit/fmod_ret are allowed. What > about those two? I don't see any explicit checks for iterator and > raw_tracepoint attach types in a switch below, so just checking if > they should be allowed to be sleepable? good point. tp_btf and iter don't use trampoline, so sleepable flag is ignored. which is wrong. I'll add a check to get the prog rejected. > Also seems like freplace ones are also sleeepable, if they replace > sleepable programs, right? freplace is a different program type. So it's rejected by this code already. Eventually I'll add support to allow sleepable freplace prog that extend sleepable target. But that's future. > > + > > if (prog->type == BPF_PROG_TYPE_STRUCT_OPS) > > return check_struct_ops_btf_id(env); > > > > @@ -10762,8 +10801,29 @@ static int check_attach_btf_id(struct bpf_verifier_env *env) > > if (ret) > > verbose(env, "%s() is not modifiable\n", > > prog->aux->attach_func_name); > > + } else if (prog->aux->sleepable) { > > + switch (prog->type) { > > + case BPF_PROG_TYPE_TRACING: > > + /* fentry/fexit progs can be sleepable only if they are > > + * attached to ALLOW_ERROR_INJECTION or security_*() funcs. > > + */ > > + ret = check_attach_modify_return(prog, addr); > > I was so confused about this piece... check_attach_modify_return() > should probably be renamed to something else, it's not for fmod_ret > only anymore. why? I think the name is correct. The helper checks whether target allows modifying its return value. It's a first while list. When that passes the black list applies via check_sleepable_blacklist() function. I was considering using whitelist for sleepable as well, but that's overkill. Too much overlap with mod_ret. Imo check whitelist + check blacklist for white list exceptions is clean enough. > > > + if (!ret) > > + ret = check_sleepable_blacklist(addr); > > + break; > > + case BPF_PROG_TYPE_LSM: > > + /* LSM progs check that they are attached to bpf_lsm_*() funcs > > + * which are sleepable too. > > + */ > > + ret = check_sleepable_blacklist(addr); > > + break;