Hi, When IO_URING_F_NONBLOCK is set on io_kiocb req->flag in io_write() or io_read() IOCB_NOWAIT is set for kiocb when passed it to the respective rw_iter callback. This sets REQ_NOWAIT for underlaying I/O. The result is low level driver always sees block layer request as REQ_NOWAIT even if user has submitted request with nowait = 0 e.g. fio nowait=0. That is not consistent behaviour with other fio ioengine such as libaio as it will issue non REQ_NOWAIT request with REQ_NOWAIT:- libaio nowait = 0:- null_blk: fio:null_handle_rq 1288 *NOWAIT=FALSE* REQ_OP_WRITE libaio nowait = 1:- null_blk: fio:null_handle_rq 1288 *NOWAIT=TRUE* REQ_OP_WRITE * Without this patch with fio ioengine io_uring :- --------------------------------------------------- iouring nowait = 0:- null_blk: fio:null_handle_rq 1288 *NOWAIT=TRUE* REQ_OP_WRITE iouring nowait = 1:- null_blk: fio:null_handle_rq 1288 *NOWAIT=TRUE* REQ_OP_WRITE * With this patch with fio ioengine io_uring :- --------------------------------------------------- iouring nowait = 0:- null_blk: fio:null_handle_rq 1307 *REQ_NOWAIT=FALSE* WRITE iouring nowait = 1: null_blk: fio:null_handle_rq 1307 *REQ_NOWAIT=TRUE* WRITE Instead of only relying on IO_URING_F_NONBLOCK blindly in io_read() and io_write(), also make sure io_kiocb->io_rw->flags is set to RWF_NOWAIT before we mark kiocb->ki_flags = IOCB_NOWAIT. Below is the deatailed testing log. -ck Chaitanya Kulkarni (1): io_uring: honor I/O nowait flag for read/write io_uring/rw.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) $ sh test-iouring-no-wait * Without this patch :- ####################### -------------------libaio nowait = 0-------------------------- ++ grep -e ioengine -e rw -e nowait fio/randwrite-libaio-nowait-0.fio ioengine=libaio rw=randwrite nowait=0 <----------------------------- overwrite=0 ++ fio fio/randwrite-libaio-nowait-0.fio --filename=/dev/nullb0 ++ grep err RANDWRITE: (groupid=0, jobs=1): err= 0: pid=40835: Thu Apr 20 21:31:09 2023 ++ dmesg -c [17978.613789] null_blk: disk nullb0 created [17978.613814] null_blk: module loaded __________________________________________________________________________ [17978.796560] null_blk: fio:null_handle_rq 1288 *NOWAIT=FALSE* REQ_OP_WRITE -------------------libaio nowait = 1-------------------------- ++ blkdiscard /dev/nullb0 ++ grep -e ioengine -e rw -e nowait fio/randwrite-libaio-nowait-1.fio ioengine=libaio rw=randwrite nowait=1 <----------------------------- overwrite=0 ++ fio fio/randwrite-libaio-nowait-1.fio --filename=/dev/nullb0 ++ grep err RANDWRITE: (groupid=0, jobs=1): err= 0: pid=40842: Thu Apr 20 21:31:09 2023 ++ dmesg -c __________________________________________________________________________ [17979.019595] null_blk: fio:null_handle_rq 1288 *NOWAIT=TRUE* REQ_OP_WRITE -------------------iouring nowait = 0-------------------------- ++ blkdiscard /dev/nullb0 ++ grep -e ioengine -e rw -e nowait fio/randwrite-iouring-nowait-0.fio ioengine=io_uring rw=randwrite nowait=0 <----------------------------- overwrite=0 ++ fio fio/randwrite-iouring-nowait-0.fio --filename=/dev/nullb0 ++ grep err RANDWRITE: (groupid=0, jobs=1): err= 0: pid=40849: Thu Apr 20 21:31:10 2023 ++ dmesg -c __________________________________________________________________________ [17979.242849] null_blk: fio:null_handle_rq 1288 *NOWAIT=TRUE* REQ_OP_WRITE -------------------iouring nowait = 1-------------------------- ++ blkdiscard /dev/nullb0 ++ grep -e ioengine -e rw -e nowait fio/randwrite-iouring-nowait-1.fio ioengine=io_uring rw=randwrite nowait=1 <----------------------------- overwrite=0 ++ fio fio/randwrite-iouring-nowait-1.fio --filename=/dev/nullb0 ++ grep err RANDWRITE: (groupid=0, jobs=1): err= 0: pid=40856: Thu Apr 20 21:31:10 2023 ++ dmesg -c __________________________________________________________________________ [17979.454102] null_blk: fio:null_handle_rq 1288 *NOWAIT=TRUE* REQ_OP_WRITE * With this patch :- #################### -------------------iouring nowait = 0-------------------------- + blkdiscard /dev/nullb0 + grep -e ioengine -e rw -e nowait fio/randwrite-iouring-nowait-0.fio ioengine=io_uring rw=randwrite nowait=0 <----------------------------- overwrite=0 + fio fio/randwrite-iouring-nowait-0.fio --filename=/dev/nullb0 + grep err RANDWRITE: (groupid=0, jobs=1): err= 0: pid=2788: Thu Apr 20 23:35:40 2023 + dmesg -c __________________________________________________________________________ [ 164.255136] null_blk: fio:null_handle_rq 1307 *REQ_NOWAIT=FALSE* WRITE -------------------iouring nowait = 1-------------------------- + blkdiscard /dev/nullb0 + grep -e ioengine -e rw -e nowait fio/randwrite-iouring-nowait-1.fio ioengine=io_uring rw=randwrite nowait=1 <----------------------------- overwrite=0 + fio fio/randwrite-iouring-nowait-1.fio --filename=/dev/nullb0 + grep err RANDWRITE: (groupid=0, jobs=1): err= 0: pid=2795: Thu Apr 20 23:35:41 2023 + dmesg -c __________________________________________________________________________ [ 164.467420] null_blk: fio:null_handle_rq 1307 *REQ_NOWAIT=TRUE* WRITE -- 2.40.0
