On 22/01/2020 06:16, Jens Axboe wrote: > On 1/21/20 8:09 PM, Dmitry Sychov wrote: >> Thank you for quick reply! Yes I understand that I need a sort of >> serializable-level isolation >> when accessing the rings - I hope this could be done with a simple >> atomic cmp-add after optimistic write ring update. > > That's not a bad idea, that could definitely work, and would be more > efficient than just grabbing a lock. > If I got it right, it still will spam the system with atomics. There is another pattern to consider, (seen in the networking world a lot). Just one thread gets completions (i.e. calls io_uring_enter()), and than distributes jobs to a thread pool. And for this distribution there are a lot of way to do it efficiently. E.g. see internal techniques in java fork join merge. That's for completion part. > Could also be made to work quite nicely with restartable sequences. I'd > love to see liburing grow support for smarter sharing of a ring, that's > really where that belongs. > >> Correct me if I'am wrong, but from my understanding the kernel can >> start to pick up newly written Uring jobs >> without waiting for the "io_uring_enter" user level call and that's >> why we need a write barrier(so that >> the ring state is always valid for the kernel), else "io_uring_enter" >> could serve as a write barrier itself as well... > > By uring jobs, you mean SQEs, or submission queue entries? The kernel > only picks up what you ask it to, it won't randomly just grab entries > from the SQ ring unless you do an io_uring_enter() and tell it to > consume N entries. The exception is if you setup the ring with > IORING_SETUP_SQPOLL, in which case the kernel will maintain a submission > thread. For that case, yes, the kernel can pickup an entry as soon as > the SQ tail is updated by the application. > -- Pavel Begunkov
Attachment:
signature.asc
Description: OpenPGP digital signature
