On 4/19/23 17:25, Jens Axboe wrote:
After bumping the flags to 64-bits, we now have two holes in io_kiocb.
The best candidate for moving is poll_refs, as not to split the task_work
related items.
Signed-off-by: Jens Axboe <axboe@xxxxxxxxx>
---
include/linux/io_uring_types.h | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h
index 84f436cc6509..4dd54d2173e1 100644
--- a/include/linux/io_uring_types.h
+++ b/include/linux/io_uring_types.h
@@ -535,6 +535,9 @@ struct io_kiocb {
* and after selection it points to the buffer ID itself.
*/
u16 buf_index;
+
+ atomic_t poll_refs;
poll wake is often done by a random CPU, would be great to limit
cache sharing, i.e. the 2nd cache line is never touched by
io_poll_wake() and it'd only need the poll entry and
the 3rd line with io_task_work().
There is one place that doesn't care about it, i.e.
clearing REQ_F_[SINGLE,DOUBLE]_POLL in poll wake, but
it's a place to improve.
+
u64 flags;
struct io_cqe cqe;
@@ -565,9 +568,8 @@ struct io_kiocb {
__poll_t apoll_events;
};
atomic_t refs;
- atomic_t poll_refs;
- struct io_task_work io_task_work;
unsigned nr_tw;
+ struct io_task_work io_task_work;
/* for polled requests, i.e. IORING_OP_POLL_ADD and async armed poll */
union {
struct hlist_node hash_node;
--
Pavel Begunkov
