On 3/9/22 4:49 PM, Artyom Pavlov wrote: > Greetings! > > A common approach for multi-threaded servers is to have a number of > threads equal to a number of cores and launch a separate ring in each > one. AFAIK currently if we want to send an event to a different ring, > we have to write-lock this ring, create SQE, and update the index > ring. Alternatively, we could use some kind of user-space message > passing. > > Such approaches are somewhat inefficient and I think it can be solved > elegantly by updating the io_uring_sqe type to allow accepting fd of a > ring to which CQE must be sent by kernel. It can be done by > introducing an IOSQE_ flag and using one of currently unused padding > u64s. > > Such feature could be useful for load balancing and message passing > between threads which would ride on top of io-uring, i.e. you could > send NOP with user_data pointing to a message payload. So what you want is a NOP with 'fd' set to the fd of another ring, and that nop posts a CQE on that other ring? I don't think we'd need IOSQE flags for that, we just need a NOP that supports that. I see a few ways of going about that: 1) Add a new 'NOP' that takes an fd, and validates that that fd is an io_uring instance. It can then grab the completion lock on that ring and post an empty CQE. 2) We add a FEAT flag saying NOP supports taking an 'fd' argument, where 'fd' is another ring. Posting CQE same as above. 3) We add a specific opcode for this. Basically the same as #2, but maybe with a more descriptive name than NOP. Might make sense to pair that with a CQE flag or something like that, as there's no specific user_data that could be used as it doesn't match an existing SQE that has been issued. IORING_CQE_F_WAKEUP for example. Would be applicable to all the above cases. I kind of like #3 the best. Add a IORING_OP_RING_WAKEUP command, require that sqe->fd point to a ring (could even be the ring itself, doesn't matter). And add IORING_CQE_F_WAKEUP as a specific flag for that. -- Jens Axboe
