On 11/4/20 7:50 AM, Jens Axboe wrote:
> On 11/4/20 4:21 AM, Stefan Metzmacher wrote:
>> Hi David,
>>
>>> New to io_uring but can't find this answer online, so reaching out.
>>>
>>> I was trying out io_uring with netperf - tcp stream sockets - and
>>> noticed a submission is called complete even with a partial send
>>> (io_send(), ret < sr->len). Saving the offset of what succeeded (plus
>>> some other adjustments) and retrying the sqe again solves the problem.
>>> But the issue seems fundamental so wondering if is intentional?
>>
>> I guess this is just the way it is currently.
>>
>> For Samba I'd also like to be sure to never get short write to a socket.
>>
>> There I'd like to keep the pipeline full by submitting as much sqe's as possible
>> (without waiting for completions on every single IORING_OP_SENDMSG/IORING_OP_SPLICE)
>> using IOSQE_IO_DRAIN or IOSQE_IO_LINK and maybe IOSQE_ASYNC or IORING_SETUP_SQPOLL.
>>
>> But for now I just used a single sqe with IOSQE_ASYNC at a time.
>>
>> Jens, do you see a way to overcome that limitation?
>>
>> As far as I understand the situation is completely fixed now and
>> it's no possible to get short reads and writes for file io anymore, is that correct?
>
> Right, the regular file IO will not return short reads or writes, unless a
> blocking attempt returns 0 (or short). Which would be expected. The send/recvmsg
> side just returns what the socket read/write would return, similarly to if you
> did the normal system call variants of those calls.
>
> It would not be impossible to make recvmsg/sendmsg handle this internally as
> well, we just need a good way to indicate the intent of "please satisfy the
> whole thing before return".
>
Attached patch handles the full send request; sendmsg can be handled
similarly.
I take your comment to mean there should be an sq flag to opt-in to the
behavior change? Pointers to which flag set?
From 9d6ca280512d3a539c771879d82645a0f7b5a27d Mon Sep 17 00:00:00 2001
From: David Ahern <dsahern@xxxxxxxxx>
Date: Mon, 2 Nov 2020 18:31:00 -0700
Subject: [PATCH] io_uring: Handle incomplete sends for stream sockets
Signed-off-by: David Ahern <dsahern@xxxxxxxxx>
---
fs/io_uring.c | 19 +++++++++++++++++--
1 file changed, 17 insertions(+), 2 deletions(-)
diff --git a/fs/io_uring.c b/fs/io_uring.c
index aae0ef2ec34d..d15511d1e284 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -421,6 +421,7 @@ struct io_sr_msg {
int msg_flags;
int bgid;
size_t len;
+ size_t offset;
struct io_buffer *kbuf;
};
@@ -4149,7 +4150,8 @@ static int io_send(struct io_kiocb *req, bool force_nonblock,
if (unlikely(!sock))
return ret;
- ret = import_single_range(WRITE, sr->buf, sr->len, &iov, &msg.msg_iter);
+ ret = import_single_range(WRITE, sr->buf + sr->offset, sr->len, &iov,
+ &msg.msg_iter);
if (unlikely(ret))
return ret;;
@@ -4171,8 +4173,18 @@ static int io_send(struct io_kiocb *req, bool force_nonblock,
if (ret == -ERESTARTSYS)
ret = -EINTR;
- if (ret < 0)
+ if (ret < 0) {
req_set_fail_links(req);
+ } else if (ret > 0 && sock->type == SOCK_STREAM) {
+ if (unlikely(ret < sr->len)) {
+ pr_debug("req %px sr->offset %lu sr->len %lu ret %d\n",
+ req, sr->offset, sr->len, ret);
+ sr->len -= ret;
+ sr->offset += ret;
+ return -EAGAIN;
+ }
+ ret += sr->offset;
+ }
__io_req_complete(req, ret, 0, cs);
return 0;
}
@@ -6460,6 +6472,9 @@ static int io_init_req(struct io_ring_ctx *ctx, struct io_kiocb *req,
/* same numerical values with corresponding REQ_F_*, safe to copy */
req->flags |= sqe_flags;
+ if (req->opcode == IORING_OP_SEND || req->opcode == IORING_OP_SENDMSG)
+ req->sr_msg.offset = 0;
+
if (!io_op_defs[req->opcode].needs_file)
return 0;
--
2.25.1
