On Tue, Jul 25, 2023 at 09:56:50AM -0400, Willem de Bruijn wrote:
> Breno Leitao wrote:
> > On Mon, Jul 24, 2023 at 06:58:04PM -0400, Willem de Bruijn wrote:
> > > Breno Leitao wrote:
> > > > Add support for getsockopt command (SOCKET_URING_OP_GETSOCKOPT), where
> > > > level is SOL_SOCKET. This is leveraging the sockptr_t infrastructure,
> > > > where a sockptr_t is either userspace or kernel space, and handled as
> > > > such.
> > > >
> > > > Function io_uring_cmd_getsockopt() is inspired by __sys_getsockopt().
> > > >
> > > > Differently from the getsockopt(2), the optlen field is not a userspace
> > > > pointers. In getsockopt(2), userspace provides optlen pointer, which is
> > > > overwritten by the kernel. In this implementation, userspace passes a
> > > > u32, and the new value is returned in cqe->res. I.e., optlen is not a
> > > > pointer.
> > > >
> > > > Important to say that userspace needs to keep the pointer alive until
> > > > the CQE is completed.
> > > >
> > > > Signed-off-by: Breno Leitao <leitao@xxxxxxxxxx>
> > > > ---
> > > > include/uapi/linux/io_uring.h | 7 ++++++
> > > > io_uring/uring_cmd.c | 43 +++++++++++++++++++++++++++++++++++
> > > > 2 files changed, 50 insertions(+)
> > > >
> > > > diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h
> > > > index 9fc7195f25df..8152151080db 100644
> > > > --- a/include/uapi/linux/io_uring.h
> > > > +++ b/include/uapi/linux/io_uring.h
> > > > @@ -43,6 +43,10 @@ struct io_uring_sqe {
> > > > union {
> > > > __u64 addr; /* pointer to buffer or iovecs */
> > > > __u64 splice_off_in;
> > > > + struct {
> > > > + __u32 level;
> > > > + __u32 optname;
> > > > + };
> > > > };
> > > > __u32 len; /* buffer size or number of iovecs */
> > > > union {
> > > > @@ -79,6 +83,7 @@ struct io_uring_sqe {
> > > > union {
> > > > __s32 splice_fd_in;
> > > > __u32 file_index;
> > > > + __u32 optlen;
> > > > struct {
> > > > __u16 addr_len;
> > > > __u16 __pad3[1];
> > > > @@ -89,6 +94,7 @@ struct io_uring_sqe {
> > > > __u64 addr3;
> > > > __u64 __pad2[1];
> > > > };
> > > > + __u64 optval;
> > > > /*
> > > > * If the ring is initialized with IORING_SETUP_SQE128, then
> > > > * this field is used for 80 bytes of arbitrary command data
> > > > @@ -729,6 +735,7 @@ struct io_uring_recvmsg_out {
> > > > enum {
> > > > SOCKET_URING_OP_SIOCINQ = 0,
> > > > SOCKET_URING_OP_SIOCOUTQ,
> > > > + SOCKET_URING_OP_GETSOCKOPT,
> > > > };
> > > >
> > > > #ifdef __cplusplus
> > > > diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c
> > > > index 8e7a03c1b20e..16c857cbf3b0 100644
> > > > --- a/io_uring/uring_cmd.c
> > > > +++ b/io_uring/uring_cmd.c
> > > > @@ -166,6 +166,47 @@ int io_uring_cmd_import_fixed(u64 ubuf, unsigned long len, int rw,
> > > > }
> > > > EXPORT_SYMBOL_GPL(io_uring_cmd_import_fixed);
> > > >
> > > > +static inline int io_uring_cmd_getsockopt(struct socket *sock,
> > > > + struct io_uring_cmd *cmd)
> > > > +{
> > > > + void __user *optval = u64_to_user_ptr(READ_ONCE(cmd->sqe->optval));
> > > > + int optname = READ_ONCE(cmd->sqe->optname);
> > > > + int optlen = READ_ONCE(cmd->sqe->optlen);
> > > > + int level = READ_ONCE(cmd->sqe->level);
> > > > + void *koptval;
> > > > + int err;
> > > > +
> > > > + err = security_socket_getsockopt(sock, level, optname);
> > > > + if (err)
> > > > + return err;
> > > > +
> > > > + koptval = kmalloc(optlen, GFP_KERNEL);
> > > > + if (!koptval)
> > > > + return -ENOMEM;
> > >
> > > This will try to kmalloc any length that userspace passes?
> >
> > Yes, this value is coming directly from userspace.
> >
> > > That is unnecessary ..
> > > > +
> > > > + err = copy_from_user(koptval, optval, optlen);
> > > > + if (err)
> > > > + goto fail;
> > > > +
> > > > + err = -EOPNOTSUPP;
> > > > + if (level == SOL_SOCKET) {
> > > > + err = sk_getsockopt(sock->sk, level, optname,
> > > > + KERNEL_SOCKPTR(koptval),
> > > > + KERNEL_SOCKPTR(&optlen));
> > >
> > > .. sk_getsockopt defines a union of acceptable fields, which
> > > are all fairly small.
> >
> > Right, and they are all I need for SOL_SOCKET level for now.
> >
> > > I notice that BPF added BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN to
> > > work around the issue of pre-allocating for the worst case.
> >
> > I am having a hard time how to understand how
> > BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN gets the MAX_OPTLEN. Reading this
> > function, it seems it is conditionally get_user().
> >
> >
> > #define BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN(optlen)
> > ({
> > int __ret = 0;
> > if (cgroup_bpf_enabled(CGROUP_GETSOCKOPT))
> > get_user(__ret, optlen);
> > __ret;
> > })
> >
> > That said, how is BPF_CGROUP_GETSOCKOPT_MAX_OPTLEN being used to
> > workaroundthe pre-allocating for the worst case?
> >
> > > But that also needs to deal woth other getsockopt levels.
> >
> > Right, and we also have a similar kmalloc() on the setsockopt path
> > (SOCKET_URING_OP_SETSOCKOPT).
> >
> > What about if I pass the userspace sockptr (USER_SOCKPTR) to the
> > {g,s}etsockopt callback directly, instead of kmalloc() in io_uring(), as
> > I was doing int the RFC[1]? It avoids any extra kmalloc at all.
>
> That looks like a great solution to me.
>
> Avoids the whole problem of kmalloc based on untrusted user input.
>
> > Something as:
> >
> > static inline int io_uring_cmd_getsockopt(struct socket *sock,
> > struct io_uring_cmd *cmd)
> > {
> > void __user *optval = u64_to_user_ptr(READ_ONCE(cmd->sqe->optval));
> > int optlen = READ_ONCE(cmd->sqe->optlen);
> > int optname = READ_ONCE(cmd->sqe->optname);
> > int level = READ_ONCE(cmd->sqe->level);
> > int err;
> >
> > err = security_socket_getsockopt(sock, level, optname);
> > if (err)
> > return err;
> >
> > if (level == SOL_SOCKET) {
> > err = sk_getsockopt(sock->sk, level, optname,
> > USER_SOCKPTR(optval),
> > KERNEL_SOCKPTR(&optlen));
> > if (err < 0)
> > return err;
> > return optlen;
> > }
>
> Do you have a plan to extend this to other levels?
>
> No need to implement immediately, but it would be good to know
> whether it is feasible to extend the current solution when the
> need (inevitably) shows up.
Yes, I plan to extend getsockopt() to all levels, but it means we need
to convert proto_ops->setsockopt to uset sockptr_t instead of
userpointers. This might require some intrusive changes, but totally
doable.
