Hi Pavel, Yes, [1] is what I mean. In an event loop every CQE is handled by a new iteration in the loop, this is the "expensive" part. Less CQEs, less iterations. It is nice to see possible kernel performance gains [2] as well, but I suggested this specifically in the case of event loops. Can you elaborate on “handling links from the user side”? [2] https://lore.kernel.org/io-uring/56a18348-2949-e9da-b036-600b5bb4dad2@xxxxxxxxx/#t -- Hielke de Vries On Sat, Apr 18, 2020, at 15:50, Pavel Begunkov wrote: > On 4/18/2020 3:49 PM, H. de Vries wrote: > > Hi, > > > > Following up on the discussion from here: https://twitter.com/i/status/1234135064323280897 and https://twitter.com/hielkedv/status/1250445647565729793 > > > > Using io_uring in event loops with IORING_FEAT_FAST_POLL can give a performance boost compared to epoll (https://twitter.com/hielkedv/status/1234135064323280897). However we need some way to manage 'in-flight' buffers, and IOSQE_BUFFER_SELECT is a solution for this. > > > > After a buffer has been used, it can be re-registered using IOSQE_BUFFER_SELECT by giving it a buffer ID (BID). We can also initially register a range of buffers, with e.g. BIDs 0-1000 . When buffer registration for this range is completed, this will result in a single CQE. > > > > However, because (network) events complete quite random, we cannot re-register a range of buffers. Maybe BIDs 3, 7, 39 and 420 are ready to be reused, but the rest of the buffers is still in-flight. So in each iteration of the event loop we need to re-register the buffer, which will result in one additional CQE for each event. The amount of CQEs to be handled in the event loop now becomes 2 times as much. If you're dealing with 200k requests per second, this can result in quite some performance loss. > > > > If it would be possible to register multiple buffers by e.g. chaining multiple SQEs that would result in a single CQE, we could save many event loop iterations and increase performance of the event loop. > > I've played with the idea before [1], it always returns only one CQE per > link, (for the last request on success, or for a failed one otherwise). > Looks like what you're suggesting. Is that so? As for me, it's just > simpler to deal with links on the user side. > > It's actually in my TODO for 5.8, but depends on some changes for > sequences/drains/timeouts, that hopefully we'll push soon. We just need > to be careful to e.g. not lose CQEs with BIDs for IOSQE_BUFFER_SELECT > requests. > > [1] > https://lore.kernel.org/io-uring/1a9a6022-7175-8ed3-4668-e4de3a2b9ff7@xxxxxxxxx/ > > -- > Pavel Begunkov >
