On 4/25/23 15:42, Ming Lei wrote:
On Tue, Apr 25, 2023 at 07:31:10AM -0600, Jens Axboe wrote:
On 4/24/23 8:50?PM, Ming Lei wrote:
On Mon, Apr 24, 2023 at 08:18:02PM -0600, Jens Axboe wrote:
On 4/24/23 8:13?PM, Ming Lei wrote:
On Mon, Apr 24, 2023 at 08:08:09PM -0600, Jens Axboe wrote:
On 4/24/23 6:57?PM, Ming Lei wrote:
On Mon, Apr 24, 2023 at 09:24:33AM -0600, Jens Axboe wrote:
On 4/24/23 1:30?AM, Ming Lei wrote:
On Thu, Apr 20, 2023 at 12:31:35PM -0600, Jens Axboe wrote:
Add an opdef bit for them, and set it for the opcodes where we always
need io-wq punt. With that done, exclude them from the file_can_poll()
check in terms of whether or not we need to punt them if any of the
NO_OFFLOAD flags are set.
Signed-off-by: Jens Axboe <axboe@xxxxxxxxx>
---
io_uring/io_uring.c | 2 +-
io_uring/opdef.c | 22 ++++++++++++++++++++--
io_uring/opdef.h | 2 ++
3 files changed, 23 insertions(+), 3 deletions(-)
diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index fee3e461e149..420cfd35ebc6 100644
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -1948,7 +1948,7 @@ static int io_issue_sqe(struct io_kiocb *req, unsigned int issue_flags)
return -EBADF;
if (issue_flags & IO_URING_F_NO_OFFLOAD &&
- (!req->file || !file_can_poll(req->file)))
+ (!req->file || !file_can_poll(req->file) || def->always_iowq))
issue_flags &= ~IO_URING_F_NONBLOCK;
I guess the check should be !def->always_iowq?
How so? Nobody that takes pollable files should/is setting
->always_iowq. If we can poll the file, we should not force inline
submission. Basically the ones setting ->always_iowq always do -EAGAIN
returns if nonblock == true.
I meant IO_URING_F_NONBLOCK is cleared here for ->always_iowq, and
these OPs won't return -EAGAIN, then run in the current task context
directly.
Right, of IO_URING_F_NO_OFFLOAD is set, which is entirely the point of
it :-)
But ->always_iowq isn't actually _always_ since fallocate/fsync/... are
not punted to iowq in case of IO_URING_F_NO_OFFLOAD, looks the naming of
->always_iowq is a bit confusing?
Yeah naming isn't that great, I can see how that's bit confusing. I'll
be happy to take suggestions on what would make it clearer.
Except for the naming, I am also wondering why these ->always_iowq OPs
aren't punted to iowq in case of IO_URING_F_NO_OFFLOAD, given it
shouldn't improve performance by doing so because these OPs are supposed
to be slow and always slept, not like others(buffered writes, ...),
can you provide one hint about not offloading these OPs? Or is it just that
NO_OFFLOAD needs to not offload every OPs?
The whole point of NO_OFFLOAD is that items that would normally be
passed to io-wq are just run inline. This provides a way to reap the
benefits of batched submissions and syscall reductions. Some opcodes
will just never be async, and io-wq offloads are not very fast. Some of
Yeah, seems io-wq is much slower than inline issue, maybe it needs
to be looked into, and it is easy to run into io-wq for IOSQE_IO_LINK.
There were attempts like this one from Hao (CC'ed)
https://lore.kernel.org/io-uring/20220627133541.15223-5-hao.xu@xxxxxxxxx/t/
Not sure why it got stalled, but maybe Hao would be willing
to pick it up again.
--
Pavel Begunkov
