The return value from `cgroup_bpf_enabled(CGROUP_GETSOCKOPT)` can change between the invocations of `BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN` and `BPF_CGROUP_RUN_PROG_GETSOCKOPT`. If `cgroup_bpf_enabled(CGROUP_GETSOCKOPT)` changes from "false" to "true" between the invocations of `BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN` and `BPF_CGROUP_RUN_PROG_GETSOCKOPT`, `BPF_CGROUP_RUN_PROG_GETSOCKOPT` will receive an -EFAULT from `__cgroup_bpf_run_filter_getsockopt(max_optlen=0)` due to `get_user()` was not reached in `BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN`. Scenario shown as below: `process A` `process B` ----------- ------------ BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN enable CGROUP_GETSOCKOPT BPF_CGROUP_RUN_PROG_GETSOCKOPT (-EFAULT) To prevent this, invoke `cgroup_bpf_enabled()` only once and cache the result in a newly added local variable `enabled`. Both `BPF_CGROUP_*` macros in `do_sock_getsockopt` will then check their condition using the same `enabled` variable as the condition variable, instead of using the return values from `cgroup_bpf_enabled` called by themselves as the condition variable(which could yield different results). This ensures that either both `BPF_CGROUP_*` macros pass the condition or neither does. Fixes: 0d01da6afc54 ("bpf: implement getsockopt and setsockopt hooks") Co-developed-by: Yanghui Li <yanghui.li@xxxxxxxxxxxx> Signed-off-by: Yanghui Li <yanghui.li@xxxxxxxxxxxx> Co-developed-by: Cheng-Jui Wang <cheng-jui.wang@xxxxxxxxxxxx> Signed-off-by: Cheng-Jui Wang <cheng-jui.wang@xxxxxxxxxxxx> Signed-off-by: Tze-nan Wu <Tze-nan.Wu@xxxxxxxxxxxx> --- Chagnes from v1 to v2: https://lore.kernel.org/all/20240819082513.27176-1-Tze-nan.Wu@xxxxxxxxxxxx/ Instead of using cgroup_lock in the fastpath, invoke cgroup_bpf_enabled only once and cache the value in the newly added variable `enabled`. `BPF_CGROUP_*` macros in do_sock_getsockopt can then both check their condition with the new variable `enable`, ensuring that either they both passing the condition or both do not. Chagnes from v2 to v3: https://lore.kernel.org/all/20240819155627.1367-1-Tze-nan.Wu@xxxxxxxxxxxx/ Hide cgroup_bpf_enabled in the macro, and some modifications to adapt the coding style. Chagnes from v3 to v4: https://lore.kernel.org/all/20240820092942.16654-1-Tze-nan.Wu@xxxxxxxxxxxx/ Add bpf tag to subject, and Fixes tag in body. --- include/linux/bpf-cgroup.h | 15 ++++++++------- net/socket.c | 5 +++-- 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h index fb3c3e7181e6..5afa2ac76aae 100644 --- a/include/linux/bpf-cgroup.h +++ b/include/linux/bpf-cgroup.h @@ -390,20 +390,20 @@ static inline bool cgroup_bpf_sock_enabled(struct sock *sk, __ret; \ }) -#define BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN(optlen) \ +#define BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN(optlen, enabled) \ ({ \ int __ret = 0; \ - if (cgroup_bpf_enabled(CGROUP_GETSOCKOPT)) \ + enabled = cgroup_bpf_enabled(CGROUP_GETSOCKOPT); \ + if (enabled) \ copy_from_sockptr(&__ret, optlen, sizeof(int)); \ __ret; \ }) #define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, optlen, \ - max_optlen, retval) \ + max_optlen, retval, enabled) \ ({ \ int __ret = retval; \ - if (cgroup_bpf_enabled(CGROUP_GETSOCKOPT) && \ - cgroup_bpf_sock_enabled(sock, CGROUP_GETSOCKOPT)) \ + if (enabled && cgroup_bpf_sock_enabled(sock, CGROUP_GETSOCKOPT)) \ if (!(sock)->sk_prot->bpf_bypass_getsockopt || \ !INDIRECT_CALL_INET_1((sock)->sk_prot->bpf_bypass_getsockopt, \ tcp_bpf_bypass_getsockopt, \ @@ -518,9 +518,10 @@ static inline int bpf_percpu_cgroup_storage_update(struct bpf_map *map, #define BPF_CGROUP_RUN_PROG_SOCK_OPS(sock_ops) ({ 0; }) #define BPF_CGROUP_RUN_PROG_DEVICE_CGROUP(atype, major, minor, access) ({ 0; }) #define BPF_CGROUP_RUN_PROG_SYSCTL(head,table,write,buf,count,pos) ({ 0; }) -#define BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN(optlen) ({ 0; }) +#define BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN(optlen, enabled) ({ 0; }) #define BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock, level, optname, optval, \ - optlen, max_optlen, retval) ({ retval; }) + optlen, max_optlen, retval, \ + enabled) ({ retval; }) #define BPF_CGROUP_RUN_PROG_GETSOCKOPT_KERN(sock, level, optname, optval, \ optlen, retval) ({ retval; }) #define BPF_CGROUP_RUN_PROG_SETSOCKOPT(sock, level, optname, optval, optlen, \ diff --git a/net/socket.c b/net/socket.c index fcbdd5bc47ac..0b465dc8a789 100644 --- a/net/socket.c +++ b/net/socket.c @@ -2363,6 +2363,7 @@ int do_sock_getsockopt(struct socket *sock, bool compat, int level, int optname, sockptr_t optval, sockptr_t optlen) { int max_optlen __maybe_unused; + bool enabled __maybe_unused; const struct proto_ops *ops; int err; @@ -2371,7 +2372,7 @@ int do_sock_getsockopt(struct socket *sock, bool compat, int level, return err; if (!compat) - max_optlen = BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN(optlen); + max_optlen = BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN(optlen, enabled); ops = READ_ONCE(sock->ops); if (level == SOL_SOCKET) { @@ -2390,7 +2391,7 @@ int do_sock_getsockopt(struct socket *sock, bool compat, int level, if (!compat) err = BPF_CGROUP_RUN_PROG_GETSOCKOPT(sock->sk, level, optname, optval, optlen, max_optlen, - err); + err, enabled); return err; } -- 2.45.2