On 2024/12/17 10:46, Jens Axboe wrote: >> Of note about io_uring: if writes are submitted from multiple jobs to >> multiple queues, then you will see unaligned write errors, but the >> same test with libaio will work just fine. The reason is that io_uring >> fio engine IO submission only adds write requests to the io rings, >> which will then be submitted by the kernel ring handling later. But at >> that time, the ordering information is lost and if the rings are >> processed in the wrong order, you'll get unaligned errors. > > Sorry, but this is woefully incorrect. > > Submissions are always in order, I suspect the main difference here is > that some submissions would block, and that will certainly cause the > effective issue point to be reordered, as the initial issue will get > -EAGAIN. This isn't a problem on libaio as it simply blocks on > submission instead. Because the actual issue is the same, and the kernel > will absolutely see the submissions in order when io_uring_enter() is > called, just like it would when io_submit() is called. I did not mean to say that the processing of requests in each queue/ring is done out of order. They are not. What I meant to say is that multiple queues/rings may be processed in parallel, so if sequential writes are submitted to different queues, the BIOs for these write IOs may endup being issued out of order to the zone. Is that an incorrect assumption ? Reading the io_uring code, I think there is one work item per ring and these are not synchronized. > > If you stay within the queue limits of the device, then there should be > no reordering. Unless the kernel side prevents non-blocking issues for > some reason. > > It's either that, or something being broken with REQ_NOWAIT handling for > zoned writes. > -- Damien Le Moal Western Digital Research
