On Tue, Aug 11, 2020 at 04:56:37PM -0400, Jeff Moyer wrote: > Jens Axboe <axboe@xxxxxxxxx> writes: > > > So it seems to me like the file state is consistent, at least after the > > run, and that this seems more likely to be a fio issue with short > > read handling. > > Any idea why there was a short read, though? Yes. See: https://lore.kernel.org/linux-xfs/20200807024211.GG2114@xxxxxxxxxxxxxxxxxxx/T/#maf3bd9325fb3ac0773089ca58609a2cea0386ddf It's a race between the readahead io completion marking pages uptodate and unlocking them, and the io_uring worker function getting woken on the first page being unlocked and running the buffered read before the entire readahead IO completion has unlocked all the pages in the IO. Basically, io_uring is re-running the IOCB_NOWAIT|IOCB_WAITQ IO when there are still pages locked under IO. This will happen much more frequently the larger the buffered read (these are only 64kB) and the readahead windows are opened. Essentially, the io_uring buffered read needs to wait until _all_ pages in the IO are marked up to date and unlocked, not just the first one. And not just the last one, either - readahead can be broken into multiple bios (because it spans extents) and there is no guarantee of order of completion of the readahead bios given by the readahead code.... Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx
